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Executive Summary 



Secondary career and technical education (CTE) is a field in transition. It is moving from a 
primary focus on preparing students for entry-level employment to preparing them for continuing 
education and training as well as employment. The rapid pace of change in technology and the 
global economy has created a demand for workers who are able to learn and adapt, and CTE 
must prepare its students to meet these demands. Greater emphasis is being placed on assessment 
to improve accountability and to verify that students have acquired the skills to undertake these 
challenges. These higher expectations come at a time when more students are taking CTE 
courses and fewer CTE teachers are graduating from teacher education programs. The field has 
responded by recruiting more teachers from business and industry, but those who enter teaching 
in this way typically have had little pedagogical training. Neither these teachers nor many of 
their colleagues who enter the profession through a traditional teacher education program are 
prepared to use technical skills assessment data to help students gain higher levels of 
competence. 

The National Research Center for Career and Technical Education (NRCCTE) is responding to 
these developments with a number of projects, some of which are being conducted by its own 
staff and others that are being directed by institutions that are partners in the NRCCTE 
consortium. Two of the projects are developing professional development models for improving 
the skills of secondary CTE teachers. The Southern Regional Education Board (SREB) is 
developing and testing an induction model for alternatively certified teachers; that is, those who 
have not completed a traditional teacher education program. NOCTI (formerly the National 
Occupational Competency Testing Institute) is applying its expertise to a professional 
development model designed to improve the ability of secondary-level CTE teachers and 
administrators to interpret data from technical skill assessments to improve instruction. 

Alternative Certification 

One of the most important challenges facing CTE is the need to build a high-quality teaching 
force. The new demands and responsibilities on CTE teachers range from integrating grade-level 
literacy and numeracy to support increased student achievement to designing intellectually 
challenging projects and real-world problems that will engage an increasingly diverse population 
of learners. Alternative routes to CTE teacher licensure, embraced for nearly 100 years as a 
viable way of transitioning those with highly valued industry experience into the teaching 
profession, are one strategy for meeting the demand for more and better CTE teachers. Although 
an increasing percentage of teachers are entering the teaching profession through alternative 
routes, the requirements for these pathways vary greatly, and a debate continues to rage as to 
whether alternatively certified teachers are less or equally effective as traditionally prepared 
teachers in impacting student achievement. 

In partnership with the NRCCTE, SREB is developing an induction model for new CTE teachers 
pursuing an alternative route to certification that will increase their competence, self-efficacy, 
and career commitment. The model is designed to build the capacity of beginning CTE teachers 
to offer instruction that is intellectually demanding and standards-focused and thus more likely to 
improve CTE students’ academic achievement. The model also builds CTE teachers’ capacity to 
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design instruction that is actively engaging using strategies like project-based learning and 
cooperative learning. Students who are actively engaged intellectually and emotionally in their 
high school courses are more likely to stay in school and graduate on time and less likely to need 
developmental (remedial) courses at the postsecondary level. 

The induction model includes 196 hours of professional development delivered through a 10-day 
summer institute prior to the first year of teaching; three, two-day workshops during the first 
year; and a 10-day summer institute following the first year. In addition, the model includes the 
support of coaching from the professional development instructor, on-site guidance from a 
mentor and administrator, and participation in an electronic community of practice. An iterative 
development process is being used to design the model. In Chapter 2, this report presents the 
findings of the first round of field tests of four training modules that comprise the professional 
development component of the model. The content of the modules was field tested between June 
2009 and February 2010 in a series of four sessions each including three, six -hour days of 
training. Two of the four field test sessions were held in Oklahoma and two were held in South 
Carolina. A total of 46 teachers participated, representing different levels of education, work 
experience, and CTE content areas. 

An analysis of the field test data provided clear indicators as to changes needed in program 
materials to meet the needs of alternative route teachers. Many learning activities were 
revised to fit the audience in order to provide more time for reflection or to clarify content. 

Field test participants identified key elements of the modules that they felt would be 
necessary for new teachers prior to entering the classroom, including: (a) the use of rubrics, 

(b) formative and summative assessment, (c) how to use a table of specifications to align 
their instructional goals and assessments to technical standards and 21st century skills, (d) 
getting to know students, (e) engaging students in developing classroom rules and 
procedures, and (f) classroom management scenarios. Data suggested that three strategies 
used by program developers were particularly effective in supporting participant learning: 

(a) use of examples in participants’ content areas, (b) use of “floating” one-on-one and 
small group coaching during cooperative learning segments, and (c) facilitated small group 
discussion in the afternoon or evening to structure reflection. Field testing on the model 
will continue through 2012, when the fully-developed model will be ready for rigorous 
experimental testing. 

Use of Technical Assessment Data 

The term “data-driven decision making” has become ubiquitous in education, and yet it seems to 
be most often discussed with reference to policy decisions related to reporting requirements and 
accountability. With the increasing emphasis on the use of standardized tests for reporting on 
school, teacher, and student achievement, the true purpose of testing — that of program or 
instructional improvement — could be lost or buried under the need to use the results for reporting 
and rating purposes. An understanding of assessment data, including their interpretation and 
many uses, can encourage teachers who have used data for classroom improvement to continue 
to do so and help those who have not used assessment data see the value in using them for 
classroom improvement beyond the meeting of regulatory requirements. 




During the first year of the NOCTI project, teachers and administrators were surveyed to 
determine if and how they use analyses of testing data to modify curriculum and instruction, how 
they learned to use data, the effectiveness of their prior training, and the topics on which they 
would like to increase their knowledge and skills. The survey results defined the felt needs of 
respondents. Short case studies were also conducted with districts that demonstrated extensive 
use of test data to identify the changes that were made in how instruction was planned and 
delivered. 

The findings from the first year, as well as an exhaustive literature review, were used to develop 
a professional development program to improve the ability of secondary CTE teachers and 
administrators to interpret assessment data to guide instructional improvement. The program, 
entitled CTEDDI (Career and Technical Educators Using a Data-Driven Improvement Model) 
includes training in analyzing data from the participants’ own students and ongoing mentoring 
and coaching. Educators are also provided access to an electronic professional sharing site where 
they become members of a community of practice. This program was field tested in nine 
locations, and information collected from the facilitators and trainees was used to identify 
changes needed to improve the materials and process. The methods and findings from the first 
two years of this project are presented in detail in Chapter 3 of this report. The revised version is 
undergoing additional review in new sites, and planning has begun to roll out the model through 
technical assistance to states that elect to provide this professional development to state- selected 
groups of teachers. Using test results in this way is one of the core components of continuous 
quality improvement and strongly encouraged by the accountability requirements and legislative 
intent of Perkins IV. 

The projects discussed in this report respond to core needs of the field, but the professional 
development challenge is far more extensive than what these projects alone address. Secondary 
CTE serves a large segment of secondary students and must contribute to their academic as well 
as technical learning. Most CTE teachers will need considerable professional development to 
broaden their teaching skills and to learn to use data for program improvement. The professional 
development they receive should be directly related to the courses they teach and of sufficient 
intensity and duration to influence their instruction. In the present economic climate, providing 
adequate time for effective professional development may be the most difficult challenge of all. 




Chapter 1 



Where We Are 

In December 1998, the American Vocational Association changed its name to the Association 
for Career and Technical Education (ACTE). This new name was chosen to reflect changes that 
had been occurring in the field during the last two decades of the 20 th century. In 2000, Lynch 
published a paper in which he identified forces acting on vocational education that led members 
of the largest association representing the field to conclude that the name of their association 
should be changed. He grouped these forces into four categories: (a) the new economy, (b) 
public expectations, (c) cognitive research on learning, and (d) school reform. The convergence 
of these factors motivated secondary-level career and technical education (CTE) to increase the 
emphasis it placed on preparing its students for postsecondary education. In the 13 years since 
the new name was adopted, the influences that Lynch identified have continued and intensified. 
In this report, we examine the expanded expectations for CTE and discuss their implications for 
professional development. 

Much more is expected of today’s CTE teachers, especially those at the secondary level, than 
was the case in the past. For most of its history, vocational education prepared its students for 
entry-level employment in jobs that required specialized skills below those at the professional 
and managerial levels and usually did not require a bachelor’s degree. Most vocational students 
felt no need to continue their education after high school. In the 1980s, the influences identified 
by Lynch converged to create a demand for skills at a higher level than could be taught in high 
school. Offshoring and automation eliminated millions of low-skill, repetitive jobs and created a 
demand for workers who could use new technology and continually learn and adapt to changes 
that could not be foreseen but were certain to occur. Employers found it increasingly difficult to 
hire workers with the skills they needed, and pressures mounted for education to raise standards 
and increase rigor. In 1990, these pressures were reflected in the reauthorization of the federal 
vocational education legislation, the Carl D. Perkins Vocational and Technical Education 
Improvement Act, which added the attainment of academic skills as one of the outcomes by 
which vocational programs were to be evaluated. Improving students’ academic skills became a 
mandated responsibility of vocational education programs in public institutions that received 
federal funds, as virtually all do. 

The two subsequent reauthorizations of the federal legislation in 1998 and 2006 strengthened this 
emphasis on improving academic skills. The current legislation (Perkins IV) requires every 
recipient of federal funds to offer at least one program of study (POS) that includes rigorous 
career and technical content aligned with challenging academic standards and leads to an 
industry-recognized credential or certificate at the postsecondary level, or an associate or 
baccalaureate degree (P.L. 109-270. Sec. 122[c] [ 1 ]). The first performance indicator for CTE 
programs required by the legislation is student attainment of “challenging academic content 
standards” and “academic achievement standards,” as adopted by the state in accordance with No 
Child Left Behind (Sec 113[B][2][A][1]). 

The stress on academics in secondary CTE will increase as states begin to implement the 
Common Core State Standards that have been developed by the National Governors Association 
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Center for Best Practices and the Council of Chief State School Officers (2010a, 2010b). The 
governors and chief state school officers of 48 of the 50 states, plus the District of Columbia, 
supported the development of these standards, and as of March 2011, 41 states and the District 
have officially adopted them. The standards for English Language Arts (ELA) and Literacy 
explicitly state: 

The Standards insist that instruction in reading, writing, speaking, listening, and language 
be a shared responsibility within the school. . . . The grades 6-12 standards are divided 
into two sections, one for ELA and the other for history/social studies, science, and 
technical subjects. This division reflects the unique, time-honored place of ELA teachers 
in developing students’ literacy skills while at the same time recognizing that teachers in 
other areas must have a role in this development as well. 

Part of the motivation behind the interdisciplinary approach to literacy promulgated by 
the Standards is extensive research establishing the need for college and career ready 
students to be proficient in reading complex informational text independently in a variety 
of content areas, (p. 4) 

Standards are specified for reading and writing in science and technical subjects for Grades 6-8, 
9-10, and 11-12. The standards for mathematics, however, do not address the responsibility of 
other content areas to contribute to learning nor do they establish different standards for Grades 9 
through 12. Instead they set forth standards for six different “conceptual categories” that high 
school students should understand and be able to use. 

Tied closely to the adoption of standards are measures to ensure these standards are being met. 
The way in which Perkins IV links its first performance indicator (attainment of challenging 
academic content and achievement standards) to the standards established by NCLB is one of the 
most prominent signs of how CTE has been affected by the calls for increased accountability in 
all of education. The second indicator is attainment of technical skill proficiencies as measured 
by assessments that are aligned with industry-recognized standards. When the results from such 
measures are properly analyzed, they can provide guidance for program and instructional 
improvement. The NOCTI study discussed in this report has developed methods to improve the 
ability of teachers and administrators to use technical assessment results for these purposes. 

Shortage of CTE Teachers 

The requirements in Perkins IV and the Common Core State Standards expand what is expected 
of CTE teachers at a time when anecdotal evidence suggests that many school districts are 
finding it difficult to hire anyone with the occupational knowledge and experience needed to 
teach the classes they offer. We are unaware of any national estimates of the number of CTE 
teachers needed in the various occupational areas, but a number of key indicators imply 
widespread shortages. Among the most prominent of these are the concerns of the state directors 
for CTE. These officials have administrative responsibility for all CTE programs in their states 
that receive federal funds, and thus have daily contact with the problems facing the field. In 
2009, their concerns led the National Association of State Directors of Career Technical 
Education Consortium (NASDCTEc) to publish Teacher Shortage Undermines CTE. This 
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publication cited the difficulties schools experience in finding qualified CTE teachers and 
identified three main factors as producing the shortage: increased enrollments in CTE courses, 
the decline in four- year teacher preparation programs, and the number of teachers reaching 
retirement age. 

The increased enrollments in CTE are due mainly to the number of secondary school students. 
This generation of students has been referred to as the “baby-boom echo,” the children of women 
who were bom during the original 1946 to 1964 boom. In the 1980s, the number of annual births 
increased to levels last seen during the original baby boom, and these levels have continued to 
the most recent year for which data are available (U.S. Census Bureau, 2009). Various national 
surveys have found that virtually all students take at least one CTE course and about 20% of 
students take several (Levesque et al., 2008). Analyses of trend data indicate that, despite the 
increase in academic requirements during the last two decades of the 20 th century, the percentage 
of high school students who take CTE courses has remained relatively steady. Larger high school 
enrollments yield larger CTE enrollments. 

Bruening et al. (2001) documented the decline in the number of institutions offering teacher 
education in CTE during the 1990s. More recent data are not available, but most observers think 
the number of such programs continued to decline during the past 10 years. Those that remain 
are producing fewer new teachers than in the past. Programs that previously had primarily 
prepared CTE teachers responded to the increased demand for corporate trainers and began 
emphasizing human resource development instead. Their graduates entered business and industry 
rather than the classroom. This shift was signaled by a change in the name of the council that 
includes most of the universities that offer doctoral programs in CTE. This council was 
established in 1976 to increase the visibility and influence of vocational education in higher 
education and adopted the name the University Council for Vocational Education. In 1998, as the 
number of students seeking degrees in human resource development increased, it changed its 
name to the University Council for Workforce and Human Resource Education (UCWHRE; 
Reynaldo Martinez, personal communication, October 27, 2010). 

The evidence on the extent to which retirement contributes to the CTE teacher shortage is more 
questionable. Earlier in the decade, Ingersoll (2003) found that among all elementary and 
secondary teachers, retirement accounted for only about one-fourth (23.7%) of teachers leaving 
education. Ingersoll analyzed data from four nationally representative surveys of schools 
conducted between 1987 and 2000 by the National Center for Education Statistics (NCES). He 
found that there was an adequate overall supply of teachers, but added that there could be 
shortages in certain content areas. Shortages, Ingersoll concluded, were primarily due to high 
teacher turnover associated with job dissatisfaction and the appeal (e.g., higher salaries) of 
nonteaching jobs. The one analysis in which he presented results for CTE teachers showed their 
annual turnover rate (13.4%) to be comparable to that of all teachers (14.3%). 

Another major indicator of the shortage of CTE teachers is the increase in the number with 
alternative certification. In some occupational areas, especially the construction and mechanical 
trades and health, alternative certification has been the primary method of entering teaching. 
Since the earliest years of vocational education, several years of occupational experience has 
been required of all new teachers. In recent years, however, alternative certification has increased 
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in occupational areas previously dominated by graduates of four- year programs. We have been 
unable to find any national estimates of the number of alternatively certified CTE teachers, but 
Bottoms and McNally (2005) provided information based on teachers in high schools in 30 states 
that are part of the High Schools That Work (HSTW) network. A survey of 12,000 such teachers 
who were hired in the five years from 2000 to 2004 found only 25% were graduates of teacher 
education programs. The high schools in the HSTW network are engaged in an intensive effort to 
improve the rigor of their CTE offerings, but three-fourths of the new teachers these schools 
hired did not complete traditional programs. 

As the number of alternately certified teachers increases, the debate about their effectiveness 
continues. Advocates for traditional four-year programs (e.g., Darling-Hammond, 2002, 2009) 
stress the need for a sound grounding in human development and pedagogy to effectively guide 
students’ learning. Proponents of alternative certification point to an emerging body of research 
(e.g., Constantine et al., 2009) that typically finds few or no significant differences in the tested 
performance of students taught by alternatively and traditionally certified teachers. All of the 
research on this topic that we could locate was conducted with elementary and secondary 
teachers in general education, not with CTE teachers. We did locate three studies of traditional 
and alternatively certified agricultural education teachers (Duncan & Ricketts, 2008; Rocca & 
Washburn, 2005; Wash, Lovedahl, & Paige, 2000), but they examined self-reported 
characteristics of the teachers (e.g., feelings of self-efficacy and participation in professional 
development), not student outcomes. Regardless of the merit of the research and arguments on 
both sides, alternative certification is needed if more CTE classes are to be offered. 

Secondary CTE thus finds itself facing a unique set of challenges. Its teachers must have 
qualifications that can be acquired only through gaining several years of occupational 
experience. Those who are recruited from business and industry often do not have four- year 
degrees, and even those who have degrees typically have little or no pedagogical training. 
Nevertheless, they teach classes with disproportionate numbers of students with special needs 
and below average academic skills (Levesque et al., 2008), and they are expected to improve 
their students’ academic skills in addition to teaching occupational skills. To compound the 
challenges, there are fewer teacher education programs in higher education that offer the kind of 
preparation needed to respond to these expanded expectations. 

NRCCTE Projects 

The National Research Center for Career and Technical Education (NRCCTE) cannot respond to 
all of these challenges, but two of its partners are conducting professional development projects 
that are responding to some. The first is being conducted by the Southern Regional Education 
Board (SREB). The principal investigator for this project, Dr. Gene Bottoms, originated the High 
Schools That Work (HSTW) initiative at SREB in 1987 and has directed it since. SREB is 
currently assisting over 1,200 high schools and career centers in 30 states and the District of 
Columbia to implement the HSTW principles. 

Data collected from HSTW schools indicate that the majority of new CTE teachers have not 
completed a traditional, four- year teacher education program If these new entrants are to become 
effective teachers, they need support and preparation in how to manage a classroom and to plan, 
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deliver, and assess instruction. The SREB project has developed an induction model that 
provides such preparation and also includes mentoring, instructional coaching, guidance from 
school principals, and an electronic community of practice. In Chapter 2, we present the rationale 
and research underlying this model, describe how it was developed and field tested, the findings 
that emerged from the field tests, and the implications of these findings for the preparation of 
alternatively certified CTE teachers. 

The second project is being conducted by NOCTI (formerly the National Occupational 
Competency Testing Institute). NOCTI is applying its expertise to a project designed to improve 
the ability of secondary-level CTE teachers and administrators to interpret data from technical 
skill assessments to improve instruction. During the first year of this project, teachers and 
administrators were surveyed to determine if and how they use analyses of testing data to modify 
curriculum and instruction, how they learned to use data, the effectiveness of their prior training, 
and the topics on which they would like to increase their knowledge and skills. Short case studies 
were conducted with districts that demonstrated extensive use of test data to identify the changes 
that were made in how instruction was planned and delivered. 

The findings from the first year were used to develop a professional development program to 
improve the ability of secondary CTE teachers and administrators to interpret testing data to 
guide instructional improvement. The program includes training in analyzing data from the 
participants’ own students and on-going mentoring and coaching. This program was field tested 
in nine locations, and information collected from the facilitators and trainees was used to identify 
changes needed to improve the materials and process. The methods and findings from the first 
two years of this project are presented in more detail in Chapter 3. 

We conclude this report by discussing how the two NRCCTE projects relate to the increased 
expectations for secondary CTE and the mandates of Perkins IV. All indicators imply that more 
and more teachers will be recruited from business and industry and need alternative certification. 
The accountability requirements of Perkins IV involve increased attention to technical skill 
assessments and the use of the data from these measures. The SREB and NOCTI projects will 
provide assistance in both of these areas. We also briefly discuss teachers’ need for content that 
is directly related to the courses they teach and for opportunities to collaborate and reflect on 
their practice with other teachers. The current economic climate may make it increasingly 
difficult to provide time for professional development of sufficient intensity and duration to be 
effective. 
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Chapter 2 



Professional Development for CTE Teachers Entering the Profession through Alternative 

Routes 

CTE teachers who are well trained and supported in entering the teaching profession are more 
likely to remain in the field and build their teaching skills over time, resulting in better 
instruction and improved learning (Hunt & Carroll, 2003; Joerger, 2003; Joerger & Bremer; 
2001). In partnership with the NRCCTE, SREB is developing an induction model for new CTE 
teachers pursuing an alternative route to certification that will increase their instructional 
competence, self-efficacy, and career commitment. The model is designed to build the capacity 
of beginning CTE teachers to offer instruction that is intellectually demanding and standards- 
focused and thus more likely to improve CTE students’ academic achievement. The model also 
builds CTE teachers’ capacity to design instruction that is actively engaging using strategies such 
as project-based learning and cooperative learning. Students who are actively engaged 
intellectually and emotionally in their high school courses are more likely to stay in school, 
acquire high school diplomas in four years, and enter postsecondary institutions without the need 
for remediation (Castellano, Stringfield, Stone, & Wayman, 2003). This chapter describes the 
iterative development of an induction model and the findings of the field testing of four training 
modules that comprise the professional development component of the model. 

Rationale 

Increasing secondary enrollment in CTE programs, the declining number and size of traditional 
CTE teacher preparation programs, and the growing number of teacher retirements have created 
a concern about the lack of supply of CTE teachers (DeWitt, 2010; NASDCTEc, 2009). To 
compound this supply challenge, high-quality CTE teaching in the 21 st century has placed new 
demands and responsibilities on CTE teachers, from integrating grade-level literacy and 
numeracy that will support increased student achievement to designing intellectually challenging 
projects and real-world problems that will engage an increasingly diverse population of learners. 
Research is needed to identify the best strategies for bringing teachers into the field, helping 
them make a successful transition to teaching, and encouraging their long-term commitment to 
the profession. 

When new CTE teachers lack crucial skills, they often become so discouraged by the complexity 
of the work and lack of formal and informal organizational supports that they leave the 
profession (Hunt & Carroll, 2003; Joerger, 2003). The U.S. Department of Education published a 
study on teacher attrition and mobility that estimated that 25% of all new teachers leave within 
the first three years (Marvel, Lyter, Peltola, Strizek, & Morton, 2006). The ultimate problem 
resulting from poorly trained CTE teachers with inadequate school support and subsequent high 
rates of teacher attrition is that CTE students will not receive engaging, academically rigorous 
instruction, increasing the probability that they will drop out (Castellano et al. , 2003). 

Alternative routes to CTE teacher licensure, embraced for nearly 100 years as a viable way of 
transitioning those with highly valued industry experience into the teaching profession, are one 
strategy for meeting the demand for more and better CTE teachers. The requirements for these 
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pathways vary greatly (Zirkle, Martin, & McCaslin, 2007) and a debate continues to rage as to 
whether alternative route teachers are less or equally effective as traditionally prepared teachers 
in impacting student achievement (Constantine et al., 2009; Darling-Hammond, 2009). For 
alternatively certified CTE teachers to make a successful transition to teaching and meet the 
demands of preparing students for further learning and the workplace, sufficient on-going 
support is needed. Induction experiences, the professional development and support activities 
that are designed to help teachers in the first few years of teaching, can provide the additional 
support that alternatively certified teachers need to meet the challenges of CTE teaching (Joerger 
& Bremer, 2001; Ruhland & Bremer, 2004). 

The Need for Quality CTE Teachers 

The current policy context in CTE reflects the belief that increasing teacher quality through 
effective preparation and professional development is key to improving the academic and 
technical achievement of CTE students. In 2006, the Perkins IV legislation called for 
professional development of CTE teachers to be “high quality, sustained, intensive, and focused 
on instruction, [increasing teachers’] academic knowledge and understanding of industry 
standards.” This legislation echoed the push for improvement in teacher quality from NCLB and 
the recommendations of the National Assessment of Vocational Education that called for better 
teacher quality in CTE (Cramer, 2004; Silverberg, Warner, Fong, & Goodwin, 2004). State CTE 
leaders have identified recruiting, training, and retaining high-quality CTE teachers as a critical 
priority to meet the challenge of improved student achievement (High Schools That Work 
Board, 2007) and ACTE’s Teacher Quality Task Force lists developing stronger induction and 
mentoring programs among its top priorities (DeWitt, 2010). 

Much is required of teachers in meeting the challenge of improving students’ technical and 
academic achievement (Gray & Walter, 2001). Implementing a CTE curriculum within the 
concept of career pathways and POS requires an understanding of career development; 
supporting academic achievement means integrating high-level literacy and numeracy; and 
engaging all students in learning, including the significant percentage of students in CTE courses 
who have special learning needs, demands an understanding of sophisticated instructional 
strategies such as cooperative learning and project-based learning. Unfortunately, as noted, many 
CTE teachers are typically less academically and pedagogically prepared than teachers of other 
subjects (Cramer, 2004; Gray & Walter, 2001). Alternatively certified CTE teachers are less 
likely to have a baccalaureate degree and more likely to be farther removed from college (Gray 
& Walter, 2001). Even if CTE teachers have a postsecondary degree, they often come to teaching 
straight from the workplace; most have been out of school for a longer period of time than other 
teacher candidates. Additionally, their postsecondary focus of study may have required fewer 
academic courses (Cramer, 2004). These circumstances suggest that alternatively certified CTE 
teachers may lack that skills and confidence to integrate the level of reading, writing, and 
mathematics that students will need to succeed in school as well as the workplace. 

The Challenge of Alternative Routes to Teaching 

In the field of education as a whole, there has been an explosion in the number of teachers 
entering through alternative certification programs. All states now offer alternative routes to 
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certification, although their requirements vary. It is estimated that between 20% and 33% of all 
new teachers enter the teaching field through alternative pathways (Feistritzer, 2007; U.S. 
Department of Education, Office of Postsecondary Education, 2006; Walsh & Jacobs, 2007). 
Although alternative routes to certification seem to be filling a need that grows out of teacher 
turnover and resulting teacher shortages (Garcia & Huseman, 2009), there is disagreement about 
the quality of the preparation and effectiveness of alternatively certified teachers. Programs are 
criticized for leading to high attrition rates, particularly because teachers have no clinical 
student teaching experience (Darling-Hammond, Chung, & Frelow, 2002). Another contention 
is that there can be a negative impact on student achievement if teachers enter the classroom 
before they are adequately prepared. Recent evidence, however, suggests that there may be little 
if any difference in the effect that alternatively versus traditionally prepared teachers have on 
student achievement. A study conducted by Mathematica Policy Research found no difference 
between the mathematics and reading achievement of elementary school students whose 
teachers entered the profession through an alternative route and the achievement of students 
who had traditionally certified teachers (Constantine et al., 2009). 

Because industry experience is a valuable qualification for CTE teachers, alternative routes have 
existed for nearly 100 years in the CTE field, particularly in the areas of trade and industrial 
education and health occupations. Ruhland and Bremer (2003) found the percentage of 
alternatively certified CTE teachers to be about 28%, but the numbers may be much higher; in a 
survey of 12,000 CTE teachers at High Schools That Work sites in 30 states, 75% of teachers 
reported entering through an alternative route (Bottoms & McNally, 2005). To date, no 
experimentally designed studies exist comparing traditional versus alternatively certified CTE 
teachers’ impact on students’ academic and technical achievement. However, the increased 
demand for CTE teachers due to higher enrollment, teachers leaving the profession, and the 
decline in the number and enrollment in traditional teacher preparation programs underscores 
the need for alternative certification programs as a pathway to CTE teaching (NASDCTEc, 

2009) and these programs will likely remain a “prevalent, if not the dominant’’ route to CTE 
teaching in this century (Gray & Walter, 2001, p. xiii). 

One of the challenges in ensuring the quality of CTE teachers who are alternatively certified is 
the wide variation in the requirements and degree of support provided to teachers as they enter 
the profession through those routes. An analysis of existing alternative routes to CTE 
certification and licensure revealed that requirements for these teaching pathways vary from state 
to state and even within states (Zirkle et al., 2007) . Of the 105 alternative routes identified, 53 
required bachelor’s degrees and 32 required completion of an organized teacher preparation 
program similar to a traditional pathway. Many pathways provide newly hired CTE teachers with 
provisional certification if they have experience in the career field in which they are to teach 
(Ruhland & Bremer, 2003; Zirkle et al., 2007). As teachers begin their first year under the 
provisional certificate, they are required to complete pedagogical coursework provided by a 
university, state agency, or local district over an extended period of time. This route may or may 
not require a postsecondary degree, depending on whether one was required in the career field. 

In addition to variations in required work experience, current employment, and educational 
experience, the alternative certification pathways also vary in the requirement of induction or 
mentor programs. Only 21 of the 105 alternative routes identified required teachers to take part 
in an induction or mentoring program (Zirkle et al., 2007). 
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Needs of Teachers Who Enter the Profession through Alternative Routes 

As a consequence of entering the field through alternative routes that do not provide traditional 
pedagogical preparation, teachers may lack the knowledge, skills, and confidence required to 
plan, deliver, and manage a challenging, engaging, and meaningful learning experience for 
students. In the field of education in general, many alternatively certified teachers, although they 
tend to have high expectations and strong idealism when they begin teaching, struggle to meet 
the demands of their jobs (Honawar, 2007). Only half of the alternatively certified teachers 
surveyed in a study for Public Agenda and the National Comprehensive Center for Teacher 
Quality said they felt prepared to teach compared to more than 80% who had completed a 
traditional teacher preparation program, and 54% reported needing more time working with a 
classroom teacher during pre-service (Rochkind, Ott, Immerwahl, Doble, & Johnson, 2007). 
Fewer than half of alternatively certified teachers say they received any training in the summer 
prior to teaching (Honawar, 2007). Stone (2000, cited in Suell & Piotrowski, 2007) studied 
alternatively prepared teachers in California and found that they listed their top need as 
curriculum development, followed by classroom resources, teaching strategies, techniques for 
handling difficult students, and classroom management. 

Historically, research studies have pointed toward the unique needs of alternatively certified 
CTE teachers. Using survey data from a national stratified sample of 352 CTE teachers in 15 
states, 43% of whom were alternatively certified, Heath-Camp and Camp (1990b) found that 
CTE teachers entering teaching from business and industry with little pedagogical training 
seemed to have more problems than CTE teachers who were traditionally certified. Similarly, in 
a study investigating the nature of teacher concerns and effective induction practices of a group 
of North Carolina CTE teachers, alternative route CTE teachers were found to have more 
concerns in general than those entering from a traditional route (Kirby & LeBude, 1998). Many 
CTE teachers who were alternatively certified knew nothing about their curriculum and needed 
orientation, help, and time to learn its scope and how to prepare lessons (Heath-Camp & Camp, 
1990a). Few new CTE teachers received curriculum guides or even any feedback or evaluation 
on their work (Camp & Heath-Camp, 1991). Furthermore, beginning CTE teachers entering 
teaching from business and industry tend to be unfamiliar with lesson planning, CTE student 
organizations, the administrative red tape of schools, or student misbehavior (Heath-Camp & 
Camp, 1990b). 

More recent research found similarities between the needs of beginning CTE teachers and those 
of beginning secondary teachers in general, including the development of skills to address 
classroom management issues, learn instructional methods, motivate students, and manage 
demands on personal time and resources (Joerger & Bremer, 2001). In addition to these skills, 
the Joerger and Bremer study outlined specific topics to meet the needs of CTE teachers in the 
areas of personal management (managing time effectively); pedagogy (designing effective 
lessons and using alternative teaching methods); students (motivating and disciplining); 
curriculum (determining scope, sequence, and pace of courses); program (facility 
management); system (advocating for funding and support); and community (establishing 
support from parents). Similar to these areas, alternative route CTE teachers surveyed at High 
Schools That Work sites expressed the need for professional development in four instructional 
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categories: planning, instructional methods, assessment, and supporting students (Bottoms & 
McNally, 2005). 

Ruhland and Bremer (2004) studied traditionally and alternatively certified CTE teachers’ 
perceptions of their first year of teaching. Traditionally certified teachers were more likely to 
report they were better prepared in pedagogy; alternatively certified teachers were more likely 
to report they were better prepared in knowledge of subject matter. The alternatively certified 
teachers in the study expressed a need for additional ongoing support in two areas of classroom 
practice: classroom management and working with special needs students. These needs are 
echoed by online survey data from those who employ CTE teachers at High Schools That Work 
sites (Bottoms & McNally, 2005). Supervisors identified classroom management as the most 
prevalent major deficiency among CTE teachers employed within the last five years. More than 
half of the respondents identified teaching strategies as a weakness for new CTE teachers. 
Forty-three percent of administrators surveyed believe that newly hired CTE teachers lack the 
skills to address student diversity and special needs. 

In summary, CTE teachers who enter through alternative routes are more likely to feel 
confident about their knowledge of the career field and less likely to feel confident in their 
ability to teach that knowledge to students. Alternative route CTE teachers’ major areas of 
concern in assuming their teaching responsibilities are classroom management and students 
motivation, as well as planning instruction for special needs students, concerns echoed by the 
administrators who supervise them. Research indicates that these teachers also need 
professional development in planning, instructional methods, assessment, and how to support 
struggling students. In addition to professional development, CTE teachers who enter through 
alternative routes require support through feedback about their work, strategies for managing 
added demands on time and energy, and resources for planning and teaching. 

Quality Induction Programs for Alternatively-Certified Teachers 

In response to the needs of beginning CTE teachers and in recognition of the essential role that 
alternative certification plays in a field in which recruiting teachers with valuable work 
experience is key to maintaining and improving the quality of the teaching force, a consistent, 
high-quality approach to induction programs for alternatively certified teachers is needed. 

Joerger and Bremer defined induction as “all of the teaching and professional activities and 
events experienced by beginning teachers from the time they sign their initial teaching contracts 
until they are fully and successfully acculturated into the profession” (2001, p. v.). Induction 
programs are designed to improve the transition to teaching, increasing teaching effectiveness 
and career commitment. 

Induction programs typically focus on the basics teachers need to survive their first year of 
teaching — classroom management, obtaining resources, designing a lesson plan — as well as 
becoming familiar with the school and learning to be a reflective practitioner. Induction activities 
include on-going personal support, assessment and feedback, continuing education, and 
socialization into the profession (Joerger & Bremer, 2001). But typical induction programs 
assume prior knowledge and classroom experience associated with traditional certification routes 
and the processes and jargon used in these programs may not be appropriate for alternatively 
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certified teachers (Szuminski, 2003). Alternatively certified CTE teachers have unique needs that 
require a unique set of induction strategies. 

At beginning of their first year of teaching alternatively certified CTE teachers specifically need: 

• a mentor in the same or related instruction area; 

• a support group; 

• curriculum, resources and tips from previous instructors; 

• an orientation to career and technical student organizations; 

• more preparation time prior to the beginning of courses; and 

• access to a variety of workshops (Joerger & Bremer, 2001) 

As part of the first year of teaching, alternatively certified teachers also required continuous 
orientation that addresses all aspects of teaching, a handbook that includes resources and 
supplies, and a help hotline that provides solutions and connects them with other new and 
beginning teachers (Joerger & Bremer, 2001). 

Ruhland and Bremer (2004) asked beginning CTE teachers about factors important to them in 
deciding whether or not to continue in the teaching profession. In this study, traditionally and 
alternatively prepared teachers were equally likely to remain in the profession, but that likelihood 
depended most on the degree to which the first year of teaching was a positive experience. 
Differences between why alternatively and traditionally certified teachers were likely to remain 
in the profession were found on three factors: sense of accomplishment, availability of a 
mentoring program, and recognition and support from a supervisor. Ruhland and Bremer 
concluded that these differences may be due to a lack of self-confidence experienced by 
alternatively certified teachers in their first year of teaching, indicating a need for additional 
support. If the first year of teaching is a positive experience, CTE teachers are more likely to 
remain in the profession. 

In a study of the perceptions of alternatively certified CTE teachers toward their mentoring and 
preparation activities, Briggs and Zirkle (2009) reported that teachers valued a summer 
workshop experience prior to the first year of teaching and subsequent courses that focused on 
teacher tasks that included classroom and lab management, instruction, and making 
presentations. Visits from course instructors were also important to the beginning teachers. The 
study findings outlined teachers’ top priorities for mentoring topics, including: planning, time 
management, student assessment, ways to prevent burnout, classroom management, and working 
with the political and cultural climates of their schools and districts. Teachers perceived 
mentoring to be most useful when the assigned mentor was from a similar content area, when 
duplication of course and employment materials was reduced, when paperwork was reduced, and 
when the mentors met with mentees on a regular basis. 

Although the aforementioned research clearly indicated that beginning teachers and CTE 
teachers specifically express a need for better support in the first year of teaching, recent research 
from a study of comprehensive induction by Glazerman et al. (2008) concluded that mentoring 
and professional development do not make a significant difference in teaching practice, student 
outcomes, or career commitment. There is a vast difference in the experiences and knowledge of 
the beginning teachers who received induction services in the Glazerman study and the CTE 
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teachers for whom the proposed induction model described in this chapter is designed. Over 90% 
of the teachers in the Glazerman induction study were already certified to teach. They majored in 
education in college and participated in 1 1 or more weeks of student teaching, primarily at the 
elementary school level. Furthermore, the Glazerman comprehensive induction study focused on 
the mentor relationship and helping beginning teachers use evidence from their practice to 
recognize and implement effective instruction. The proposed model is a coherently integrated 
combination of professional development and support designed to scaffold CTE teachers’ 
learning and maximize impact on teaching practice. The selection of specific induction activities 
and the quality of their delivery are essential to the success of induction models. Briggs and 
Zirkle (2009) highlighted the problem that exists today of poorly designed mentoring and 
induction programs that lack practical and research-based topics specifically designed for CTE 
teachers. Further research is needed to inform the field about the specific induction activities that 
will ultimately result in improved teacher performance and career commitment. 

Conceptual Framework 

The proposed intervention is an induction model that targets the malleable factors associated 
with high-quality induction programs through two primary components: on-going professional 
development and support. Professional development is designed to respond to the problem of 
inadequate CTE teacher instructional competence and self-efficacy in planning and delivering 
high-quality instruction. Support elements are designed to respond to the problems of isolation 
and dissatisfaction with school culture reported by teachers that leave the profession across all 
disciplines (Borman & Dowling, 2008; Brill & McCartney, 2008). 

The induction model is designed to respond to conditions in the field. Those conditions include 
105 different routes to certification for CTE teachers (Zirkle et al., 2007), dictating a wide array 
of entry requirements from state to state. At the same time, research on teacher attrition suggests 
that between 25-75% of new teachers leave the classroom within their first three years (Bottoms 
& McNally, 2005; Marvel et al., 2006). Finally, the vision articulated in Perkins IV demands that 
CTE teachers can plan, deliver, and assess engaging instruction that (a) integrates academic 
content, especially in reading and math, (2) ties to technical concepts and standards in the 
teacher’s subject area, (c) connects with students’ interests, talents, aspirations, and broader 
program of study, (d) helps students see how coursework is tied to all aspects of their industry, 
and (e) equips students with essential 21 st -century skills. These conditions suggest the need for a 
new teacher induction model that could be adopted by states to increase the pedagogical skills of 
new teachers consistent with Perkins IV while mitigating some of the common drivers of teacher 
attrition by providing sustained school-based support to new CTE teachers in their first year. 

Prior studies have identified factors that contribute to early career teacher attrition. Those factors 
include: (a) inadequate technical instructional skill (Baldacci, 2006; Lemov, 2010), (b) 
unsupportive professional cultures (Moore Johnson & The Project for the Next Generation of 
Teachers, 2006), and (c) low confidence or sense of efficacy (Tschannen-Moran & Woolfolk 
Hoy, 2001). 

Drawing on prior research in the fields of teacher preparation and induction (Borman & 

Dowling, 2008; Brill & McCartney, 2008; Heath-Camp & Camp, 1990a; Joerger, 2003), 
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program developers adopted a basic conceptual framework for an induction model aimed to 
address teacher attrition, shown in Figure 2.1. 




Figure 2.1. Basic conceptual framework. 



Such a model has been implemented before with mixed results (Glazerman et al., 2008). 
Induction models nearly always provide professional development, although it is often not 
focused enough on technical pedagogy (Lemov, 2010); some induction models have combined 
professional development with collegial support through mentors and networking (Glazerman et 
al., 2008). To differentiate this conceptual framework - and therefore the induction model - from 
the basic framework, program developers further defined each element in terms of quality. As 
shown in Figure 2.2, it is the combination of high quality professional development and high 
quality site-based support by mentors, administrators, and coaches that program developers 
expect will yield increased levels of teacher instructional competence, self-efficacy, and career 
commitment, and therefore differentiate outcomes from this induction model from those of 
similar prior efforts. 



•Dialog with peers 
•Time to reflect 
•Opportunities to apply learning to authentic 
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•Provided over extended period of time with 
opportunities to implement learning and 
receive feedback between formal learning 
experiences 
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Figure 2.2. Differentiated conceptual framework. 



As shown in Figure 2.2, high quality professional development is defined as teacher learning 
experiences consistent with research on effective adult learning. Specifically, professional 
development must engage teachers with new content and experiences that include dialog with 
peers, application of new learning through authentic tasks, and reflection on their learning 
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(Mezirow, 1997). High quality school support is defined as regular structured weekly interaction 
between a new teacher and a qualified mentor and separate structured weekly interaction with an 
administrator; regular monthly interaction with peers through online learning communities; and 
quarterly observation and feedback from a skilled coach. Teacher instructional competence is 
operationally defined as performance in instructional planning, use of instructional strategies, 
assessment, and classroom management as measured by a validated classroom observation 
protocol. Teacher career commitment is defined as teacher self-report of intent to remain in the 
field of teaching for more than three years as measured by an instrument for assessing career 
commitment. Teacher self-efficacy is defined as the degree to which teachers feel they can 
influence students and their learning as measured by the Teacher Sense of Efficacy Scale (TSES) 
developed by Tschannen-Moran and Woolfolk Hoy (2001) and corroborated by teacher 
interviews and focus groups. 



Components of the Induction Model 



Professional Development 

Teacher professional development is among the most comprehensively researched aspects of 
the schooling enterprise. The proposed induction model draws substantially on this 
knowledge base. Sparks and Hirsh (1997) reviewed the literature and best practices in 
professional development and identified the following characteristics of training most likely 
to lead to changes in on-the-job behavior, such as the training designed for the proposed 
induction module. Those characteristics include: 

• Focused on individual and organizational development (DuFour, DuFour, & 
Eaker, 1998; Senge, 1999); 

• Aligned with school and district strategic plans (Fullan, 2001); 

• Focused on student needs and learning outcomes (DuFour et al., 1998); 

• Job-embedded (DuFour et al., 1998); 

• Facilitates teachers’ study of their own teaching and learning rather than placing 
“experts” in the role of “transmitting” knowledge (Darling-Hammond & Sykes, 
1999); 

• Focused on both content- specific and generic instructional skills (Ball & Cohen, 
1999); and 

• Involves the principal as instructional leader to sustain growth (Fullan, 2001; 
Senge, 1999; Sergiovanni, 1999). 

The interrelationship between training and organizational support is a strong theme and 
justifies the vital school support aspect of the proposed induction model. 

The framework and content for the professional development component of the induction 
model, specifically aimed at increasing new CTE teachers’ instructional competence and 
self-efficacy, was developed in the first year of the project. Four professional development 
modules were framed around the perceived needs of beginning teachers (Heath-Camp & 
Camp, 1990a, 1990b; Joerger & Bremer, 2001; Rochkind et al., 2007) and standards 
outlining what both beginning and expert teachers need to know and be able to do 
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(Danielson, 1996; Interstate New Teacher Assessment and Support Consortium [INTASC], 
1992; National Board for Professional Teaching Standards [NBPTS], 1997). 

The four professional development modules include: (a) instructional planning, (b) instructional 
strategies, (c) classroom assessment, and (d) classroom management. The framework for these 
modules is outlined in Table 2.1. These content areas respond directly to the need for new CTE 
teachers to be better prepared to deliver high quality, engaging instruction that integrates 
rigorous academic material with CTE content around intellectually demanding projects and 
activities (Hunt & Carroll, 2003; Joerger, 2003). Further, a significant component of all four 
modules focuses on assessing and addressing the diverse needs of all learners, thereby 
responding to the need for highly competent CTE teachers able to intellectually, emotionally, 
socially, and behaviorally engage all “students including special populations” (Perkins IV). 

Table 2.1 



Framework for Professional Development Modules 



Module Title and 
Description 


Module Outcomes — Areas of Teacher Instructional Competence 


Module 1: Instructional 
Planning 

Effective CTE 
instruction is carefully 
planned to target the 
academic and technical 
skills within a career 
pathway that prepare 
students for both further 
learning and the 
workplace. 


Create short-term and long-term standards-based instructional plans 
based on the varying learning needs of students. 

Specific Areas of Emphasis: 

• Plan instruction that reflects the new mission of CTE, 
supporting both college- and career-readiness 

• Set instructional goals that incorporate industry standards, 21st 
century skills, all aspects of an industry, and high-level academics 
(reading, writing, mathematics, and science) 

• Make instructional modifications for diverse learning needs 

• Plan collaboratively with colleagues, advisory committee, and 
postsecondary partners 

Reflect, both individually and collaboratively, on the effects of 
instruction and use the reflective process to continually improve 
instructional practice. 

Specific Areas of Emphasis: 

• Reflect individually with guiding questions and the use of a 
professional portfolio 

• Reflect collaboratively through the use of protocols for 
providing feedback and looking at student work 


Module 2: Instructional 
Strategies 

Research-based 
instructional strategies 
engage and motivate 
students and deepen 
learning. 


Use instructional strategies that actively engage students in learning 
and encourage the development of problem-solving, critical thinking, 
and teamwork skills. 

Specific Areas of Emphasis: 

• Use problem-based and project-based learning with real world 
place problems and tasks 

• Design intellectually demanding assignments 

• Use cooperative learning 

• Integrate academic skills, including embedded literacy and 
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Module Title and 
Description 


Module Outcomes — Areas of Teacher Instructional Competence 




numeracy 


Module 3: Classroom 
Assessment 

Assessment provides a 
clear picture of 
students’ performance 
in relation to the 
standards, informing 
teaching practice and 
further learning. 


Use formal and informal assessment strategies to evaluate student 
progress toward learning goals and provide feedback to improve 
student learning. 

Specific Areas of Emphasis: 

• Use formative and summative assessment methods that prepare 
students for workplace and postsecondary types of assessment (for 
example, employer and college-readiness exams) 

• Incorporate student self-assessment, especially through a 
portfolio of work 

• Use rubrics to clearly define assessment criteria 

• Create written exams that mirror standardized-assessment-type 
or employer-exam-type questions 

• Assess student progress in using reading, writing, and 
mathematics to solve problems and take action in the field 

• Develop a plan for grading and reporting student progress 


Module 4: Classroom 
Management 

A well-managed 
classroom centers on 
respectful, collaborative 
relationships that 
support student 
learning. 


Create a learning environment that encourages student motivation, 
positive behavior, and collaborative social interaction. 

Specific Areas of Emphasis: 

• Establish appropriate rules and routines for the CTE lab 

• Create a culturally responsive classroom 

• Offer rewards and recognition to encourage effective effort 
and increase student motivation 

• Design extra help to support all students in reaching standards 

• Communicate with parents and engaging them in supporting 
students’ success 



Concept papers were developed for each module, outlining content and the rationale for that 
content based on literature and best practice. These concept papers are included in 
Appendices 2 A through 2D. Expert panels reviewed the concept papers, ensuring that the 
content was comprehensive and appropriate for a teacher induction program. Professional 
development sessions — detailed through a guide for participants and a guide for instructors — 
were designed for each module. The instructor guide includes an overview and objectives for 
units based on topics within the module and learning activities and objectives for each lesson. 
Presentation slides and suggested artifacts to support the learning activities have also been 
developed for the instructor. The participant guide includes an overview and objectives for 
each unit (also printed in the instructor guide), handouts to support the learning activities led 
by the instructor, planning forms, suggested activities beginning teachers can do with their 
assigned mentors and building administrators, and field activities for implementing and 
reflecting on the use of the plans developed in the professional development sessions. The 
module units and lessons are outlined in Appendix 2E. 
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In addition to the content of the modules, the professional development component of the 
induction model includes a suggested sequence and delivery to affect the intended outcomes 
of instructional competence and self-efficacy. The sequence of the modules is designed to 
provide support before, during, and after first-year CTE teachers begin classroom teaching 
through three phases: (a) 10 days of intensive instruction during the summer prior to the first 
year of teaching, (b) successive nine-week segments of application and reflection through 
delivery of instruction in their own classroom, aligned with each quarter of the school year, 

(c) three two-day workshops corresponding with each quarter of the school year that focus on 
refining and deepening understandings, and (d) 10 days of structured reflection, 
reinforcement, and revision in the summer following the first year of teaching. This sequence 
responds to the inadequacy of existing models of first-year teacher preparation that fail to 
provide adequate individualized support to new CTE teachers throughout the first year in the 
classroom (Alliance for Excellent Education, 2008). Providing such support addresses three 
problems: early career teacher attrition as a result of a difficult first year (Kapadia, Coca, & 
Easton, 2007; Smith & Ingersoll, 2004), longer time-to-competency of new teachers (Villar 
& Strong, 2007), and the varying needs of the widely diverse population of adult learners that 
are CTE teachers. 

In addition to addressing the content and sequence of professional development, the induction 
model also outlines, through the instructors’ and participants’ guides for each module, specific 
delivery methods that reflect the primary principles of adult learning (Knowles, 1975; Knowles 
& Associates, 1984) and model instructional practices that teachers will be expected to use in 
their own classrooms. Instruction will incorporate cooperative learning, as well as project- and 
problem-based learning (Merrill, 2007; Schmidt, 1993). Cooperative learning provides an 
opportunity for social interaction and social construction of knowledge and skills among the 
adult learners. The module instruction is organized around projects that involve the complex 
tasks of teaching, engaging beginning teachers in problem-solving, decision making, and 
investigative activities, and providing the opportunity to create realistic products that they will 
actually use in their classrooms (Jones, Rasmussen, & Moffitt, 1997; Thomas, 2000; Thomas, 
Mergendoller, & Michaelson, 1999). 

The first summer session (10 days) includes the most essential concepts from each topic that the 
teachers need in the classroom, including curriculum and instructional planning, how to get to 
know students, and how to set the right tone. These topics have immediate relevance and 
applicability to their first weeks on the job. Teachers plan out the first nine weeks of instruction 
in some depth and craft a skeleton outline of instruction for the next nine weeks. They also 
identify a significant, authentic activity, problem, or project that would cover at least 10 days of 
instruction and involve problem-based learning. As they plan that problem, project, or activity 
unit, they identify the embedded literacy and mathematics skills and look for instructional 
strategies and methods for enhancing those components. Additionally, they learn how to assess 
students’ performance using both paper-and-pencil and performance assessments, focusing on 
technical skills, literacy and mathematics. All of these instructional design choices are made for 
the purpose of best preparing teachers for their first days and weeks on the job where they have 
an opportunity to test their new learning in the authentic environment of their classroom, 
consistent with research indicating that adults learn best when they can apply and reflect on their 
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learning (Knowles, 1975; Mezirow, 1997). 

Support 

In addition to professional development, the induction model provides beginning CTE teachers 
with the support of a mentor, coaching from the professional development instructor, the support 
of their principal or school administrator, and participation in on-going communities of practice 
through electronic conversations and guided reflection. 

Mentoring. As a result of their research, Heath-Camp, Camp, Adams-Casmus, Talbert, and 
Barber (1992) and Joerger and Bremer (2001) recommended a structured mentoring program for 
providing support and encouragement. In a literature review on beginning teacher induction, 
Serpell and Bozeman (2000) found that many researchers regard mentoring as the most critical 
component of induction programs, with teachers regarding it as one of the most helpful parts of 
induction. The review also pointed out that those new teachers who had mentors said they were 
more prepared and more likely to stay in teaching. Smith and Ingersoll (2004) found that 
teachers who had mentors in the same subject field and who collaborated with other teachers 
were more likely to stay in teaching after their first year. Mentoring relieves the isolation many 
new teachers feel and provides them with collaborative problem-solving, emotional support, 
motivation and encouragement, and information and suggestions (Joerger, 1997). The literature 
is very clear that mentors themselves must be veteran teachers who are rigorously selected; that 
there should be administrative support for the mentoring; and that contact between the mentor 
and the beginning teacher should occur at least weekly, if not daily (Allen, 2003; Burk, Ford, & 
Mann, 1996; Feiman-Nemser, Carver, Schwille, & Yusko, 1999; Feistritzer & Chester, 2000; 
Hunt & Carroll, 2003; Villar & Strong, 2007; Yoon, Duncan, Fee, Scarloss, & Shapley, 2007; 
Zeichner & Schulte, 2001). 

In the proposed induction model for CTE teachers, trained mentors address the problem of CTE 
teacher dissatisfaction with teaching and school culture (Rowley, 1999). Each new CTE teacher 
participating in the model has a mentor who is a master teacher at his or her school. All teachers 
selected as mentors participate in a two-day training session to prepare them to support new 
CTE teachers. The mentor training focuses on developing skills as mentors of CTE teachers, 
providing explicit guidance on how to differentiate mentor support to new teachers early in the 
school year and during subsequent months. Each mentor’s strengths and experiences are 
assessed and the mentor training adapted to meet the needs of the mentors. Mentors participate 
with the new teachers in the CTE induction training and in the follow-up training. In addition, 
mentors also work with the instructor and participate in the community of practice as a way to 
further strengthen and polish their skills. During the school year, mentors meet with the new 
teachers for at least 15 to 20 minutes each day for the first month and then for an hour per week 
during the rest of the school year to discuss critical issues that have arisen. They also participate 
with their beginning teacher in the electronic communities of practice and monthly webinars. 

Coaching. The induction model includes a coaching component to undergird the mid-year 
instructional support element. The coaching component of the model recognizes that the 
problems of practice new teachers encounter are not solved solely through training. Technical 
assistance and coaching are necessary to help new teachers translate their own learning about 
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how to deliver quality instruction and manage classrooms into effective classroom strategies in 
practice (Yoon et al., 2007). The instructors from the initial summer workshop also fulfill the 
role of instructional coaches for new teachers. The instructor acting as coach returns three times 
during the year before each follow-up workshop to determine how well the new teacher is 
implementing what he or she is learning and to seek evidence that the practices teachers are 
learning at each training session are being put into place. The instructor communicates with the 
mentor and local administrator prior to each school visit to discuss what would be most helpful 
to the new teacher during the visit. 

The instructor in the role of coach observes the new CTE teacher's classroom instruction, 
particularly in view of the four strands from the training, observe any gaps that need to be 
addressed, and provide a written critique with suggestions on how to continuously improve in 
each area and address gaps. The instructor in his or her role as coach seeks evidence that the 
administrator and mentor are supporting the new teacher and gives suggestions for further 
support. The instructor as coach is expected to meet with the new teacher, the mentor, and the 
administrator to engage in a professional dialogue on the new teacher's successes in the 
classroom, gaps or challenges in implementing the new knowledge and skills, and necessary 
adjustments for addressing these gaps. Finally, the instructor in the role of coach identifies issues 
and topics that can be dealt with at the follow-up weekend workshops and determine how the 
initial training can be improved and modified to better meet the needs of the beginning teachers. 
All instructors use a common format when they conduct coaching visits and a common rubric to 
describe their findings on the new teachers’ accomplishments, challenges, plans, and the 
presence and quality of support from the administrators and others. 

Communities of practice. Encouraging the development of professional networks and 
communities of practice responds to two aspects of the problem this proposed induction model is 
designed to address. The first of those problems is the instructional competence and self-efficacy 
of teachers. Engagement in communities of practice is known to contribute to meaningful adult 
learning (Mezirow, 1997), maximizing learning outcomes from the professional development 
modules. The second problem the induction model is designed to address is organizational 
support. Teachers benefit from being able to learn and grow through collaboration in 
professional learning communities (Ball & Cohen, 1999; Borko, 2004; Stone, Alfeld, & Pearson, 
2008). Sharing experiences in a group is also important for adult learning (Knowles & 
Associates, 1984). New teachers also benefit from a peer support group limited to beginning 
teachers and including meetings, listservs, and other mechanisms to discuss common 
experiences, successes, challenges, solutions, and resources (Heath-Camp et al., 1992; Joerger & 
Bremer, 2001). Communities of practice create a collegial environment that can meet teachers’ 
needs for encouragement and a sense of belonging, thereby reducing feelings of isolation that 
may lead new teachers to give up and leave the classroom. 

In the proposed induction model, instructors play a key role in building a community of practice 
around the knowledge and skills participating new teachers are developing. In addition to the 
workshops throughout the year, beginning teachers participate in electronic coaching that 
includes reviewing (on a monthly basis) new teachers’ electronic reflective journals. In these 
journals, new teachers describe what worked each week, what did not work, new insights they 
gained, where they are having difficulties, where they need help, what they plan to do in the 
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following week to try to address issues that have emerged and how they hope to build on their 
successes for the coming week. At the end of each month, teachers are asked to review their 
entries and summarize the big ideas learned over the course of the month, deficits they still need 
to address, and how they plan to address them. These journals add an important reflection for the 
teachers and a qualitative dimension that will assist the evaluation of the project implementation. 

The community of practice is further supported by listservs and blogs. These tools, in addition to 
teachers’ reflective journals and coaching visits to participants’ schools, serve to identify a list of 
difficulties or challenges that the participating new teachers are having; prioritize them; research 
their solutions; and develop a webinar (an online seminar/workshop) on one topic each month. 
During each webinar, new teachers and their mentors have 15-minute intervals to discuss how 
the new teachers can incorporate the research-based practices and strategies in their instruction to 
address the difficulties. The intent is for new teachers to walk away with a clear idea of how to 
solve the problems they encounter. 

Administrator and school system support. The problems of teacher instructional competence, 
self-efficacy, and career commitment are ultimately problems that school districts and schools 
own. New teachers especially need to feel supported by administrators and colleagues. This 
includes time allotted for preparation, collaborative planning and peer assistance, and supportive 
and timely feedback (Hunt & Carroll, 2003; Stigler & Heibert, 1999; Yoon et al., 2007; Yopp & 
Young, 1999). Indeed, research suggests that the problems of career commitment of CTE 
teachers are likely the result of school systems (Camp & Heath-Camp, 1991). The induction 
model requires buy-in and support from district and school leaders (Szuminski, 2003). Such buy- 
in ensures: 



• Teachers’ attendance at the training to learn and subsequently implement the 
practices learned is a priority for the school and district. 

• The school in which the teacher works has planned to support implementation of the 
practices learned. 

• The district is committed to supporting teachers as they attend training and return to 
the school site to apply what they learned 

• Participants in the training know why they are there and understand what they are 
expected to do to prepare for the training and what they must do when they return to 
their schools. 

The administrator support element of the induction model addresses the key aspect of ensuring 
the success of the participating CTE teachers. The designated administrator supervising the 
beginning teacher participates in two days of training during the 10-day summer institute along 
with the mentor assigned to the beginning teacher. The supervising administrator is expected to 
meet with the mentor and the new CTE teacher at least monthly to discuss implementation of 
what the teacher learns in the training. The supervising administrator is also expected to visit the 
new CTE teacher's classroom weekly for the first month (then monthly) and observe classroom 
practices, using a checklist targeted around the four strands from the training. The supervising 
administrator meets with the teacher and the mentor to provide feedback. In addition, the 
supervising administrator is expected to support the time needed for the new teacher and mentor 
teacher to meet and encouraged to be supportive in an informal way (e.g., when meeting in the 
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hallway, asking how it’s going and what support is needed). 



Methodology 

Theoretical Framework for Research Approach 

To conduct investigation of the field test, and to generate data that can be used to successively 
develop and revise the induction model, program evaluators looked for a theoretical framework 
to provide methodological guidance. The framework selected was a “design research” approach 
(Middleton, Gorard, Taylor, & Bannan-Ritland, 2008). Design research is characterized by a 
seven-phase cycle of inquiry that Middleton et al. (2008) called the “‘compleat’ design 
experiment.” The aim of the design experiment is to investigate the relationship between the 
intended function of an intervention, the design or form of the intervention, and the behavior 
resulting from the intervention. The field test reported here fits into the cycle at Phase Four, 
which involves prototyping and trials using an “iterative, progressive and disciplined” approach 
(Middleton et al., 2008, p. 32). Middleton et al. (2008) wrote, “The articulation of the 
hypothetical structure to be investigated is critical for a design experiment to be truly an 
experiment ” (p. 34). Accordingly, the aim of the inquiry is not only to generate data that can be 
used to make revisions to the teacher induction materials and delivery, but to refine the theory of 
change based on learning that emerges through field testing. 

Using this approach ensures that in successive rounds of testing and revision, program 
developers can explain how the model contributes to outcomes. This is a key departure from 
traditional approaches using experimental design and was, in part, a response to the guidelines 
for Institute for Education Sciences (IES) Goal 2 development and innovation projects (Albro, 
2010). Independent of Goal 2 guidelines, however, these methods remain the most appropriate 
for developing a “product” (a finished set of materials that comprise an induction model for new 
CTE teachers) over the course of three years for which the small numbers of teacher participants 
involved render an experimental design and/or use of inferential statistical procedures unreliable, 
inadequately nuanced, and poorly aligned to research questions. 

Purpose and Research Questions 

The induction model is being developed through three rounds of field tests. The first round 
focused on the content and delivery of the professional development modules. Subsequent years 
will include a field test of the full induction model and a state-led field test of the model to 
determine feasibility of implementation. Data from each field test will be analyzed and used to 
improve the model. In this chapter, the data from the field test of the professional development 
modules is reported. This field test occurred during the 2009-2010 school year. The field test was 
guided by three objectives: (a) to test the theory of change on which the induction model is 
based, (b) to identify content revisions to the instructional module delivered, and (c) to identify 
structural revisions to the design of the overall induction model. Six research questions guided 
the field tests: 

1. Are module materials relevant, usable and clear? If not, why? 

2. Is the scope of module content reasonable? If not, why? 
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3. Is the delivery of modules consistent with research-based adult learning principles? If 
not, why? 

4. Do teacher participants produce artifacts reflecting the intended outcomes of each 
module were achieved? If not, why? 

5. Are our assumptions of what constitutes “teacher instructional competence” 
appropriate for first and second year CTE teachers? If not, why? 

6. Do the measures used during the first year of field tests generate the kind of 
information needed to tell us that the model is working as intended? If not, how do 
they need to be revised? 

Methods and Procedures 

Evaluators developed a design to generate multiple data sources to inform each research question 
(see Table 2.2). Participants were selected by state agency partners. Program developers 
provided partners with the following criteria as guidelines for selecting participants: 

• Candidates should meet all basic requirements to participate in a state-approved route 
to alternative certification in CTE; 

• Candidates should exhibit basic mastery of the content area in which he/she will 
teach; 

• Candidates should have one or fewer years teaching experience; 

• Candidates should contribute to the diversity of the group by content area expertise, 
professional experience, postsecondary education level, expected teaching setting 
(e.g. comprehensive high school or technology center), and personal characteristics; 
and 

• [Added for Field Tests 3-4] Candidates should possess advanced mastery of basic 
literacy and numeracy. 

Description of Data Collection Methods 

Observation. Evaluators observed all four field tests, making entries in a log to record levels of 
participant engagement based on observation of body language and on- or off-task discussion. 
These logs were used to illuminate other data sources, such as the participant quick cards, to 
illuminate what was happening in the classroom at specific times throughout the field test. In 
addition, state partners (personnel from CTE divisions at the state department of education) as 
well as CTE teacher educators from local universities observed each field test and completed a 
structured observation journal. 

Quick cards. In order to capture participant response to specific segments of training with as 
much fidelity as possible, evaluators developed “quick cards” to be administered at the end of 
each segment of instruction, approximately every 60-90 minutes throughout the three days of 
training. Cards were coded with participant IDs. At the direction of evaluators, participants 
paused to “card,” requiring that they record the specific time called out by evaluators and rate the 
immediately preceding segment of training on four dimensions of adult learning quality: (a) 
relevance to their classroom, (b) opportunities for dialog with peers, (c) opportunities to apply 
learning, and (d) adequacy of time devoted to the segment. The ratings were on a scale of 1 to 6 
with 1 the lowest and 6 the highest. 
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Table 2.2 

Year 1 Research Matrix: Evaluation of Content, Scope, and Relevance 



Research Question 





Observations 


Quick Cards 


Pre-Post 

Constructed 

Response 


Pre-Post TSES 


End of Day 
Evaluation 


Teacher Focus 
Groups 


Instructor Debrie] 


Expert Panel 


Is content relevant, 
useable, clear? 


X 


XX 






X 


X 






Is the scope of content 
reasonable? 










X 


X 


X 




Is it delivered consistent 
with adult learning 
principles? 

Do artifacts reflect 
intended outcomes? 


X 


XX 


X 


XX 










Are our assumptions of 
“teacher instructional 
competence” appropriate? 
Do our measures function 
as we need them to? 












X 


X 


X 


Population/Sample 


Evaluators 

and 

Observers 




Teacher Participants 




Instructors 


Expert 

Panelists 



Pre-post tests. A pre-post test ‘battery” was administered to all participants and included three 
elements: (a) constructed response items created for each of the four field tests, based on 
expected learning outcomes defined by program developers; (b) demographic information 
including open response questions asking about participant motivation to become a teacher, and 
(c) the Teacher Self-Efficacy Scale (Tschannen-Moran & Woolfolk Hoy, 2001), a validated 
instrument that measures teacher self-efficacy using three subscales: efficacy in student 
engagement, efficacy in instructional strategies, and efficacy in classroom management. 
Evaluators experimented with different models of administration of these over the year before 
settling on the optimal arrangement of administering the pretest electronically prior to arrival at 
the training, and administering a paper-and-pencil posttest at the end of the third day of training 
to ensure maximum participation rates. Teacher self-efficacy scale results are reported in 
Appendix 2F. 

End-of-day evaluation. At the end of each day of training, participants completed an end-of-day 
evaluation that asked participants to rate the following elements on the whole for each day: 
concepts presented, binder materials, activities, and overall value. The instrument used seven 
semantic differential scales on which participants were asked to rate each element: 
clear/confusing, realistic/unrealistic, engaging/boring, relevant/pointless, useful/useless, 
organized/scattered, and challenging/easy. 



23 




Focus groups. Every teacher participated in a 60-90 minute focus group on one evening during 
the three-day training. Participants for each focus group were purposefully selected to ensure 
racial and gender diversity. The focus groups during the fourth field test were selected to 
distribute personalities that tended to dominate conversation, based on experience with that 
cohort during the third field test. Protocols were modified slightly to conform to the specific 
content of each field test; however, the purpose of the protocol was to identify areas of strength 
in the modules and areas that need improvement with a view to identifying the underlying needs 
of new CTE teachers that were either met or not met through the modules. Participant insights 
regarding sequence of content were also solicited. Focus group protocols, as well as protocols for 
the instructor debriefing are included in Appendix 2G. 

Instructor debriefs. At the end of each field test day, instructors were debriefed using a brief 
structured interview protocol designed to uncover expert assumptions, particularly those on-the- 
fly decisions that expert instructors make that deviate from planned activities. Instructors were 
also asked to identify aspects of the training that they felt were most successful and to reflect on 
what they felt teachers learned, identifying the evidence (what did they see or hear) that led them 
to their conclusions. 

Observer interviews. Observers were also interviewed at the end of each day with two questions: 
what learning objectives do you feel teachers learned today, and what did you see or hear that 
tells you they learned this? 

To address Research Question 6, a panel of national experts was convened twice to review the 
overall evaluation design and instrumentation. The panel was comprised of published scholars 
with expertise in using design research to develop teacher preparation experiences, evaluating 
large scale alternative teacher certification efforts, CTE teacher education, and general program 
evaluation. The panel provided substantial feedback that contributed to revisions to the 
instrumentation over the course of the year. 

Data Analysis 

All interviews, focus group transcripts, and constructed response items were analyzed using 
qualitative open coding (Corbin & Strauss, 2008; Miles & Huberman, 1994). Simple paired 
samples t tests were conducted on the pre-post data from the Teacher Sense of Efficacy Scale 
(Tschannen-Moran & Woolfolk Hoy, 2001). All TSES items were entered into SPSS and were 
analyzed using basic statistical tests of mean differences. 

Findings 

Analysis of data from each field test generated myriad findings that program developers used in 
successive cycles of revision and retesting over the course of the year. Selected findings that 
emerged in all four field tests are reported here. Findings fall into four categories: characteristics 
and needs of participants, strategies that enhanced participant learning, planning logistics and 
content of professional learning, and methodological findings. 
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Characteristics and Needs of Participants 

The content of the modules was field tested between June 2009 and February 2010 in a series 
of four sessions each including three, six-hour days of training. Two of the four field test 
sessions were held in Oklahoma and two were held in South Carolina. A total of 46 teachers 
participated, representing different levels of education, work experience, and CTE content 
area (see Table 2.3). 

Table 2.3 

Demographic Characteristics of Field Test Participants 



Characteristic 


n 


% 


Gender 


Male 


24 


52 


Female 


22 


48 


Race/Ethnicity 


White 


35 


76 


American Indian 


7 


15 


African American 


5 


11 


Hispanic 


1 


2 


Age 


Less than 25 


2 


4 


25-34 


17 


37 


35-44 


13 


28 


45-54 


10 


22 


55-64 


4 


8 


Highest Level of Education 


High School only 


1 


2 


High School with professional training 


13 


28 


Associate’s Degree 


5 


11 


Bachelor’s Degree 


19 


41 


Beyond Bachelor’s Degree 


8 


17 


Subject Area 


Agriculture and Natural Resources 


3 


6 


Arts, Audio, Video Technology and Communication Services 


4 


8 


Construction 


7 


15 


Education and Training Services 


2 


4 


Health Services 


9 


18 


Hospitality and Tourism 


2 


4 


Human Services 


5 


11 


Information Technology Services 


5 


11 


Legal and Protective Services 


1 


2 


Manufacturing 


3 


6 


Transportation, Distribution, and Logistics Services 


3 


6 


Scientific Research, Engineering and Technical Services 


1 


2 



Note. Some participants self-identified more than one racial/ethnic category. 
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Two findings emerged primarily from analysis of focus group transcripts regarding the 
characteristics and primary concerns of the participants in these field tests. The first finding 
speaks to the level of basic literacy and numeracy skills found within this group of alternatively 
certified CTE teachers. The second finding emerged without prompting in multiple focus groups, 
pointing to the key challenges and concerns facing these new CTE teachers. 

Oklahoma’s state policy for recruiting alternatively certified CTE teachers introduces 
virtually no barriers to entry, including no minimum score requirement on tests of basic 
skills. Accordingly, participants in the two Oklahoma field tests demonstrated a wide range 
of basic literacy and numeracy skills. Observations by instructors and guest observers 
suggested that the concepts of integrating academic content such as literacy and numeracy 
skills were especially challenging for these CTE teachers, some of whom did not have strong 
mastery of those basic skills themselves. The participants indicated awareness of this during 
focus groups. Referring to a text on project-based learning, one participant said, “There were 
a lot of large words in there that could have been re- worded in another way. I can't tell you 
those words because I didn't know the meaning of them. And that went kind of rough. A lot 
of us are not college people, okay? We worked in the field for 25-30 years. I'm just stating 
that. And some of those larger words probably need to be put in more of a layman's terms.” 

Other field test groups noted concern regarding the cognitive demand of integrating academic 
content into CTE instruction as part of the constellation of skills expected of a brand-new 
teacher, noting that teachers are not likely to be receptive to instruction in doing this until the 
second half of the first year. 

Regardless of their pre-existing levels of basic skills, all field test groups of teacher participants 
indicated that what is foremost on their minds is how to motivate students and manage their 
classrooms. One focus group participant said, “My biggest battle right now is keeping the kids 
interested. We can write rubrics until we’re blue in the face, and write lesson plans, and write 
long-range plans, and write critical maps and all this stuff. But, for whatever reason, it’s just 
keeping the kids’ interest and motivation.” The verbatim phrase, “You can lead a horse to water 
but you can’t make them drink,” came up independently in several focus groups. 

Professional Development Characteristics and Strategies that Enhanced Participant Learning 

Data suggested three strategies used by program developers were particularly effective in 
supporting participant learning: (a) use of examples in participants’ content areas, (b) use of 
“floating” one-on-one and small group coaching during cooperative learning segments, and (c) 
facilitated small group discussion in the afternoon or evening to structure reflection. Based on 
feedback from the instructors, observers, and participants, many learning activities were revised 
to fit the audience, to provide more time for reflection, or to clarify content. For example, a 
lesson was added on asking classroom questions based on comments from observers and module 
reviewers. The literacy lesson was expanded to include sample literacy assignments such as 
reading technical journals, writing each week in class, and assigning a basic report as part of a 
project. 
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Rating Level 



Participants in the first focus group raised program developers’ awareness of the importance of 
linking the content of the modules to specific examples tied to their CTE content areas. One 
participant said, “I need more specific training in the areas I teach,” whereas another participant 
stated plainly, “I really can’t use the material I learned here because it is not connected to my 
content.” Following that feedback, program developers took explicit steps to determine the 
content areas of participants in advance of subsequent field tests, and put together resource 
binders with content-specific examples for every teacher’s content area. In the focus group for 
the third field test, participant comments suggested this change was having its intended effect. 
One participant noted, “You go to other trainings and [what they present] doesn’t really apply [to 
me]. It’s overall, generalized teaching strategies. You come here and it’s reversed. Here, you sit 
down and you have people who understand what CTE teaching is. . .and say, ‘This is how you 
apply this to your classroom.’” 

With regard to coaching, several data sources suggest that teacher learning is best supported 
when there are coaches to move among small groups during cooperative learning segments. 
Participant interviews and focus groups both yielded strong agreement that this was an important 
aspect of learning for them that helped to “individualize” instruction. The quick cards show 
spikes in relevance, dialog, and application following segments where there was small group- 
coach interaction (See Figure 2.3). 




4 

9:25am 10:05am 10:50am 11:35am 1:20pm 2:03pm 3:09pm 

Relevance Dialog Practice — — Time 

Figure 2.3. Quick card ratings of adult learning quality from Day 1 of Field Test 2. 

Participants in the focus groups noted that the coaches do not have to have content expertise in 
their CTE area, but only expert knowledge in the process - whether it is rubrics, testing, or 
instructional strategies. Observers noted that although the cooperative learning strategies used 
throughout the modules are consistent with adult learning principles, they were not equally 
effective for all groups, particularly those that do not receive a visit from a coach/instructor 
during their small group discussion. 

Finally, facilitated discussion following the formal training agenda helps teachers further process 
their new knowledge. Though participants liked a brisk instructional pace, they indicated in 
focus groups that having an informal but semi- structured time to debrief, “process,” and “digest” 
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what they learned was tremendously beneficial to their learning and to facilitating connections 
among participants. During the field test, the focus groups performed this function. 

Planning Logistics and Content of Professional Learning 

Feedback from teacher-learners, as well as from state agency administrators in planning 
modules, underscores how important it will be to select optimal days and times for the three two- 
day follow-up sessions during the 2010-2011 school year. School holidays, end-of-term grading 
periods, and other school-based demands on teachers must be balanced with limited school 
resources for substitute teachers to cover participants’ absences. 

Across all four field tests, teachers identified key elements of the modules that they felt would be 
necessary for new teachers prior to entering the classroom. Those elements were segments on: 

(a) the use of rubrics, (b) formative and summative assessment, (c) how to use the table of 
specifications to align their instructional goals and assessments to technical standards and 21 st - 
century skills, (d) getting to know students, (e) engaging students in developing classroom rules 
and procedures, and (f) the twelve classroom management scenarios. 

Methodological Findings 

The expert panel reviewed the model’s design and instrumentation twice during the first year of 
field testing. Recommendations from the panelists focused on enhancing the qualitative 
methodologies to generate more descriptive data, including adding interviews of individual 
participants and adding detailed questions to protocols for observers and instructors regarding 
their observations of participant learning. 

Panelists also interrogated the use of teacher retention as a measure of program impact given the 
influence of the current economic climate and the short time frame for the project. In lieu of 
retention data, panelists recommended the use of measures of career commitment as a more 
accurate proxy for the outcome the program aims to achieve, and further suggested adding a 
school climate measure to the evaluation design to account for other, more powerful influences 
on teacher attrition. Evaluators are incorporating all of these suggestions into the evaluation 
design for the coming year. 

In addition to findings generated by the expert panelists, evaluators captured key challenges to 
implementing the design methodologically. The original design of this model included a 
selection process that would allow for basic skills testing as well as pretesting. However, the 
constraints of how new CTE teachers are hired interfered with implementation of rigorous 
selection methods. Specifically, because there is a very compressed timeline between date of hire 
and the beginning of the first 10-day summer institute offered as part of the induction model, the 
amount of time available to vet teacher applicants ranges from 0 to 10 days. It also introduces 
methodological challenges in generating a pre- measure of teaching. Accordingly, as the project 
moves forward, a cross-sectional design will be employed to compare measures of teacher 
participants early in their first year with measures of a comparison group of teachers who do not 
participate in the induction model. 
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This same challenge has implications for the underlying stance of the project in terms of what 
kinds of teachers program developers aim to support. Some programs like Teach for America use 
rigorous recruitment and selection methods to screen out all but the “best and brightest” with the 
most potential for success. The timing challenges that emerged in preparing the field test helped 
program developers to clarify that this induction model is designed to raise the bar across the 
board, for all teachers, not just those who show exceptional potential. This will have implications 
for the research design because it will be more difficult to show improvement on average. 
Accordingly, in coming years, program evaluators will focus on creating “rich, thick 
descriptions” of each individual participant to enable better analysis of how elements of the 
induction model interact with individual characteristics. 

Implications 

The year-long field testing process reported here was the beginning of a three- year effort to 
develop and refine a model of new CTE teacher induction. At this stage, it was not the intention 
of program evaluators to produce findings that could speak to the success of this training effort in 
equipping teachers with teaching competency or self-efficacy. However, early findings can still 
be of use to researchers or program developers who are undertaking similar projects to support 
new CTE teachers. Two particular findings have implications for program design and research 
methodology. 

One such finding emerged from the discovery of state practices that influence the hiring, and 
therefore the selection, of possible participants for a two-week residential summer program. 
Because implementing a meaningful selection process for participants in a summer institute 
would require identifying prospective teachers in the spring, it is likely that participants will 
either not be brand-new teachers, or that participants will not meet more rigorous selection 
criteria (because there is not enough time to “weed out” applicants who do not meet higher 
standards). Program developers can plan for either contingency depending on what conditions 
and priorities prevail in a given state. If it is the former, where participants have already 
completed a year of teaching, some elements of professional development could be eliminated, 
such as how to organize the classroom, whereas other elements could be more deeply explored, 
such as how to integrate academic content into CTE coursework. If the hiring date is late in a 
given state, it is likely that some participants will come to the summer institute with low-level 
basic skills and accommodations will need to be made in instructor vocabulary and reading and 
writing assignments for teacher participants. 

In terms of methodological implications, the individualized “rich, thick” profile approach that 
program evaluators of this induction model will take in future field testing iterations is likely the 
best research approach for any multi-state effort. The reason for this is the discovery that not 
only are there substantial differences in existing state requirements and support between states, 
but there are dramatic differences in support offered to teachers of different content areas within 
a single state. When levels of state support and training are high, it would be difficult to 
distinguish between the effects of this induction model and the effects of state support. Using a 
case study approach to explore and compare the experiences and outcomes of individual teacher 
participants in light of their content areas, years of classroom experience, years of industry 
experience, and other state- or university-provided support and preparation, among other factors, 
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is more likely to detect program influence than a large-scale statistical model until more reliable 
measures of teaching are developed that could be used as the basis of a value-added model. In 
either case, the costs of conducting such an inquiry are substantial. Selecting representative cases 
to include in an in-depth case study approach can keep costs down, although having to hire 
consultants or train principals to conduct teacher observations with adequate inter-rater reliability 
can drive costs up for a large-scale statistical approach. 

Following the field test of the full induction model in 2010-2011, two states will be selected to 
lead field tests of the full induction model with their own facilitators and a second cohort of 
beginning teachers in 2011-2012. Project staff will train these facilitators during the 2010-2011 
field test. These tests of the model will provide some idea of the feasibility of this model to be 
implemented at the state level. 

With subsequent revisions based on the data from the field tests, the resulting fully developed set 
of materials and processes for the induction model will be ready to be implemented to scale in 
2012. Results of the field test will be published with the goal of informing the field. Technical 
assistance to states, districts, and community colleges will be available on the use of the 
materials to shape or improve their own programs. 
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Chapter 3 



Professional Development for Teachers and Administrators on the Use of Assessment Data 

The term “data-driven decision making” has become ubiquitous in education, and yet it seems to 
be most often discussed with reference to policy decisions related to reporting requirements and 
accountability. Deserving at least equal attention, according to Boudett, Mumane, City, and 
Moody (2005), is the ability of teachers and administrators to use student assessment results to 
determine student skills and instructor effectiveness and then use the results to improve overall 
instruction. Yet not a great deal of research has been reported on this topic, and very little 
appears in the literature about career and technical educators’ use of assessment data, particularly 
as it relates to program and instructional decisions. Although the term assessment data can be 
very broadly defined, the primary focus of the study reported in this chapter was on standardized 
summative assessments. In addition, although several forms of data were addressed, the main 
focus was on technical assessment, relevant because it is required by Perkins IV. 

The purpose of the first year of the NOCTI study was to determine through descriptive survey 
research how secondary CTE educators use technical assessment data to improve program 
curriculum and to identify individual and group instruction needs. Of particular interest were 
how they learned to make use of the data, what specific types of professional development were 
provided if any, their perception of the effectiveness of this training, and finally what types of 
professional development they would consider most effective for the future. 

In the second year, the results of the Year 1 research were used to inform the creation of a 
professional development program for secondary CTE teachers and administrators on how to 
effectively interpret assessment data and use that information to make instructional 
improvements in the classroom. The developed intervention included the materials necessary for 
implementing the training, a facilitator/delivery system for maintaining the system within the 
school setting, and a preliminary pilot test evaluation of the effectiveness and feasibility of the 
program. The purposeful sample for the first-year survey research targeted CTE educators in five 
states (Illinois, Missouri, Oklahoma, Pennsylvania, and Virginia) and included a representative 
sample of secondary programs in both career- technical centers and comprehensive high schools. 
Within those states and schools, educators were targeted in four occupational clusters (Business 
Education, Construction, Health Science, and Manufacturing). State and cluster selection 
information is discussed in a section to follow on Year I. 

With the increasing emphasis on the use of standardized tests for reporting on school, teacher, 
and student achievement, the true purpose of testing (program and student improvement) could 
be lost or buried under the need to use the results for reporting and rating purposes. An 
understanding of assessment data, including their interpretation and uses, can encourage teachers 
who have used data for classroom improvement to continue to do so and help those who have not 
used them see the value in using test data for classroom improvement beyond mere reportage. 
This is supported by the findings in the survey; of those educators who indicated that their 
opinion of testing had improved, several commented that the main reason for the change was a 
better understanding of tests and the use of assessment data. The more teachers understand the 
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process and function of assessment, the more they are likely to see it as a tool rather than a threat 
(Cromey, 2000). 

Characteristics of the desired and needed professional development were identified by the survey 
and a continuous literature search process, as well as a series of case studies of schools’ 
successful use of data for instructional improvement. As the research indicates, effective 
professional development for teachers must be of a relatively long duration, strongly 
contextualized, collaborative, activity-based, include an emphasis on analysis and reflection, and 
be connected to comprehensive change processes focused on improving student learning (e.g., 
Darling-Hammond, Wei, Andree, Richardson, & Orphanos, 2009; Smith, Hofer, Gillespie, 
Solomon, & Rowe, 2003; Wayman, Midgley, & Stringfield, 2005). However as Smith et al. 
(2003) indicated, effective professional development alone is unlikely to result in productive 
long-term change unless other factors are taken into account, such as time and resource 
availability, organizational support, and the feasibility of implementation and continuation of 
learning within the school system. 

The goal of the intervention developed was to create and pilot professional development for CTE 
teachers on the use of technical assessment data for data-driven decision making that is both 
based on sound principles of learning and delivered in a manner that is sustainable within a 
school system. For the cycle to be complete, it is imperative that teachers be able to diagnose 
problems by looking at evidence (data) and then receive training to be able to plan instructional 
modifications around that evidence (Butler & McMunn, 2006). 

The intervention developed as part of this study involves multiple stages, illustrated in the chart 
below, which is adapted from a model suggested by the Institute of Education Sciences (IES, 

U.S. Department of Education, 2009b). Several of the stages overlap in time and include (a) the 
development and refinement of the professional development training, including content matter, 
structure of the training (including layout, exercises, etc.), delivery, and related facilitator 
materials, (b) the selection and development of measurements to assess the effectiveness of the 
learning and the successful application of that learning to the classroom, (c) the selection of 
appropriate pilot sites, (d) the selection of facilitators/coaches, (e) the administration of the 
training and coaching, and (f) the initial evaluation of the program process and effects. Figure 3.1 
below lays out the intervention, the theory of change applied, and associated resources, activities, 
outcomes, and measures. 

A total of nine secondary CTE school sites in the sample of five states were selected to pilot the 
professional development on the use of assessment data. An administrator and team of teachers 
from the targeted four clusters were identified at each site. Some of the sites participated in 
Round 1 of the pilot and others in Round 2, which was planned to start a month later so as to 
allow for the application of learning from Round 1, in consonance with the principle of 
continuous iterative refinement of the intervention. 

Facilitators selected for their ability to engage participants in the topic provided the highly 
interactive, project-based initial training. Unique aspects of this intervention are the opportunity 
for participants to work with data from their own students (via technical skills pretests donated 
by NOCTI) and to work with a cohort of peers to develop action plans that relate to their specific 
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schools and classrooms (Wayman & Cho, 2008). After the initial training, participants were 
coached over several months in their school environment by the same facilitators who provided 
the training. Educators were also encouraged to interact with others who were receiving the 
professional development, thus forming a professional learning community, which has been 
shown to be helpful for learning and growth through collaboration (Ball & Cohen, 1999; Borko, 
2004; Stone et al., 2008). 
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Figure 3.1. Simple model of change: Teacher professional development for the use of assessment 
data (adapted from the IES model of change; IES, U.S. Department of Education, 2009b). 



Year 1: Survey and Case Studies 



Purpose and Goal of Year 1 Activities 

This descriptive survey study was an investigation into how secondary CTE educators use 
technical assessment data to improve program curriculum and to identify individual and group 
instruction needs. Of particular interest was how they learned to make use of the data, what 
specific types of professional development were provided, if any, their perception of the 
effectiveness of this training, and finally what types of professional development they would 
consider most effective for the future. 



The following objectives were undertaken in this study: 



• The primary objective was to investigate the extent of and processes for CTE educator 
use of technical assessment data to inform instructional decisions and the source(s) of 
knowledge that enable them to do so. 

• The second objective was to examine the types of professional development CTE 
educators have received related to the use of data from technical assessment, to determine 
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how the offerings have been perceived by those participating, and to learn how they have 
applied them. 

• The third objective was to find out how administrators and teachers rate the effectiveness 
of types of professional development on this topic, and what characteristics of the 
professional development they judge have helped or would help them develop skills for 
making instructional improvements. Associated with this objective was the intent to do a 
review of relevant literature to gain insights about what others have reported as desirable 
professional development characteristics and to include them in the findings. 

The associated research questions, posed as hypotheses, were: 

1. Those who know more about test data interpretation tend to use the data for the purpose 
of instructional improvement aimed at higher student achievement than those who know 
less. 

2. Those who use test data for program improvement perceive an impact from the data- 
driven changes. 

3. Those who use test data for program improvement have had professional development on 
the topic. 

4. No significant differences exist between the comprehensive high school CTE responses 
and the regional CTE center program responses. 

Methodology 

The principal research strategy used was survey research with a purposeful sample of five states 
with representative examples of both secondary career-technical centers and comprehensive high 
school CTE programs. The study design involved the development and conduct of a survey 
distributed through the directors of all the secondary career-technical centers in each of the states 
and the CTE coordinator of a like number of comprehensive high schools to all the CTE 
instructors in the same four selected program areas. The number of schools in the final sample 
was 265, some of which offer all four programs and others as few as one. The rationale for 
conducting the survey at both career-technical centers and comprehensive high schools was that 
they were representative of the two types of CTE delivery institutions at the secondary level, and 
this allowed for a comparison of the data between them. That rationale is supported by NCES 
data, which indicate that 49.4% of CTE students are served through high school based programs 
and the remainder through either full-time or part-time CTE regional centers (Institute of 
Education Sciences, U.S. Department of Education, 2009a. 

The sample frame was delimited to four selected programs from the full range of programs the 
schools offered so that the data for those programs could be compared across sites. The criteria 
for selection were intended to result in a set of programs as representative as possible of the 
whole of CTE, and as non-duplicative as possible of variables such as capital intensity, program 
outcome with regard to certification, student gender preponderance, cluster, and use of 
technology. The overriding criterion was that they be commonly enough found among schools 
that the set would have a high probability of providing a large number of schools for the 
sampling frame. Further, they were to be programs in areas of at least average occupational 
growth as rated by 0*NET (U.S. Department of Labor, 2008). On that basis, the following 
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programs, with suggested occupations within the clusters, were selected, for the reasons 
indicated at that time: 

• business education (accounting) — is not capital-intensive and some segments lead to 
industry certification, some segments use high technology (accounting is in Financial 
Services, a national high growth industry that is economically critical, projected to 
add substantial numbers of new jobs, and is being transformed by technology and 
innovation); 

• construction (carpentry) — a national high growth industry that is economically 
critical, projected to add substantial numbers of new jobs, and is being transformed 
by technology and innovation (within the construction industry, carpentry has average 
expected growth) ; 

• health science (nurse assisting) — a national high growth industry that is 
economically critical, projected to add substantial numbers of new jobs, and is being 
transformed by technology and innovation (nurse assisting is a support occupation, 
which is expected to grow much faster than average); 

• manufacturing (welding) — is also a national high growth industry that is 
economically critical, projected to add substantial numbers of new jobs, and is being 
transformed by technology and innovation (welding leads to industry certification and 
is relatively capital-intensive, high technology use). 

The survey link was emailed to each local school CTE director with an introduction and a 
request to forward the email to up to two other administrators and a total of up to eight 
instructors in the selected four programs. Each message included a support letter from the CTE 
Director of that state, the purpose being to underscore the school directors’ perception of the 
study as important and enhance their motivation to distribute it and complete their own copy. 

The message also contained information about an incentive — each person who completed a 
survey received a gift certificate in thanks. Although research on the use of incentives favors 
prepaid incentives over postpaid incentives (Armstrong, 1975; Dillman, 2007), the effect of 
postpaid incentives, although smaller, can have an impact on survey response rates and 
especially dropout rates, including Internet, or online, surveys (Deutskens, Ruyter, Wetzels, & 
Oosterveld, 2004; Goritz, 2005, 2006; Heerwegh, 2006). In the marketing field, it has been found 
that payment after the fact is likely to yield more completed surveys than payment up front (D. 
Friedmen, personal communication, June 11, 2008). Follow up with non-respondents was 
accomplished by personal email and ultimately by phone. 

Target sample. One of the main criteria for selection of the states, beyond their having relevant 
types of CTE delivery systems, was that NOCTI could obtain the data efficiently because those 
in the organization have a working relationship with educators at several levels in those states. 
Professionals at NOCTI believe, based on experience, that the likelihood of obtaining quality 
responses is enhanced when existing relationships are involved. Importantly, this also meant that 
considerable time and resources could be saved over the alternative of taking a random sample 
and then having to search out the related information; thus, convenience was a factor. 
Furthermore, some randomization was possible within those states. It should be noted that 
geographical representation was not a criterion. A comprehensive sampling frame was developed 
as part of this study. 
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The states that best fit the criteria and were included in the study were Illinois, Missouri, 
Oklahoma, Pennsylvania, and Virginia. A description excerpted from the State Profile 
(NASDCTEc, 2008) of each of these states with regard to CTE follows: 

• Illinois — 26 of 666 public high schools offer solely (or primarily) CTE courses (note 
that the NCES data lists 50 CTE centers, so the definition may be substantially 
different due to academic coursework offered at the CTE center); 341,340 students 
enrolled in CTE; Illinois envisions their career clusters as organized around five 
specific career clusters: Health Occupations, Business, Marketing and Management 
Education, Industrial and Technical Education, Agricultural Education, Family and 
Consumer Sciences. 

• Missouri — 57 of 572 public high schools offer solely (or primarily) CTE courses; 
147,717 students enrolled in CTE; Missouri believes that Career Clusters provide the 
infrastructure for a seamless educational transition between all learner levels. The 
mission includes support for high quality professional development for teachers. 

• Oklahoma — 54 of 466 public high schools offer solely (or primarily) CTE courses; 

1 15,894 students enrolled in CTE; Oklahoma believes that Career Clusters offer 
many benefits and should be used as a basis for high school reform. 

• Pennsylvania — There are 81 regional career and technical schools and of the 501 
public high schools approximately 300 offer some CTE courses; 96,338 students 
enrolled in CTE; career clusters are decided upon and implemented at the local level 
based upon Pathways. 

• Virginia — 49 of 296 public high schools offer solely (or primarily) CTE courses; 
208,852 students enrolled in CTE; The State Board of Education has allowed students 
who have an industry certification/state licensure and who are CTE program 
completers to count this for up to two verified credits for graduation. 

It should be noted that all of these states are in the HSTW network, which requires considerable 
attention to assessment data, as well as other conditions, so their educators may be different in 
their responses than those in non-HSTW states. However, not all schools in each of the HSTW 
states participate in the HSTW program, so it cannot be inferred that the schools sampled pay a 
greater attention to data than in other states; ultimately, this focus depends on the voluntary 
involvement of the sample sites that the state departments of education helped select, and any 
others identified as having similar involvement in the issues studied. 

Literature review conducted to inform instrument development. The survey development was 
preceded and informed by a literature search on the topic of data-driven instructional decision 
making, particularly with reference to CTE. Further, an inventory was made of the known types 
of professional development or other assistance that were offered to educators, on this and other 
topics. These are not all specific to CTE, yet are informative. For example, the course at Harvard 
University described by Boudett et al. (2005) could generate ideas for CTE even though it was 
applied in another setting. The model recommended by Guskey and Sparks (1996) of the 
National Staff Development Council, being used for other NRCCTE work, indicates that the 
characteristics of quality professional development involve content, process, and context. These 
categories form a framework for studying the nature of the professional development that was 
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reflected in the survey. The survey was thus developed with the background of as much 
information as possible. 

Instrument development. The Principal Investigators drafted two surveys to be administered 
online, one for CTE administrators and a second for CTE teachers. The developers felt that 
beginning with separate survey forms would be a cleaner way to approach the task than to try to 
think of both pools of responders simultaneously. Starting from the types of questions submitted 
as part of the approved project proposal, the developers sought information from the literature 
review with an eye toward targeting the remaining most relevant and important questions and to 
give the surveys content validity. In the course of the development, internal sources were also 
tapped for input. 

The intent was to develop the survey so as to generate responses across the topic of the use of 
data for instructional improvement, especially the use of technical assessment data to both (a) 
improve program curriculum and (b) identify individual and group instructional needs. Of 
particular interest were survey findings that relate to how the educators learned to make use of 
data, what specific types of professional development were provided, if any, their perception of 
the effectiveness of this training, and what types of professional development they would 
consider most effective for the future. 

Significant attention was given to the best way to ask the questions and to format and design the 
instruments, with the understanding that such detail might make a difference in how willing the 
recipients would be to respond. Care was taken to limit the total time required to what the 
respondents might consider reasonable to commit to it. The manner in which items were asked 
was geared toward minimal use of open-ended questions, because open-ended questions tend to 
have a lower response rate (Darby, 2007; Dillman, 2007). 

Because contact information was needed to send the gift certificates, the surveys were not 
anonymous; however, recipients were assured that the contact information would be used only 
for the purpose of follow up and that no respondent would be identified by name in the data set 
or the resulting report unless permission was requested and granted. 

Survey Review Process 

The survey drafts were reviewed and discussed widely so as to gain team consensus on their 
efficacy to gain the desired information. The process was composed of several steps. 

Other NOCTI staff reviewed the instruments for clarity and sensibility and to get an estimate of 
the time for completion. Then the surveys were submitted for review to selected professional 
development experts in the CTE field, including other NRCCTE partners, to ascertain if the 
“right” questions were being asked. A review response form was developed to encourage some 
standardization in the form of the responses. In each stage of review, the participants were sent 
information by email about the purpose of the study, the methodology, and the objectives of and 
process for the agreed-to type of review. After receiving the detailed reviews from these experts, 
the surveys were revised and finally merged to form one survey that could be administered to 
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both administrators and teachers because the bulk of the survey could be worded so as to apply 
to both. 

A cognitive laboratory was conducted with selected survey pilot participants to better ensure that 
the meaning of the survey items was clear and interpreted in the same way by multiple 
participants, for the sake of survey reliability. The purpose was to identify any point of 
misunderstanding and obtain suggestions for improvement. For the cognitive laboratory 
methodology, participants were asked to complete a survey while talking to the facilitator and to 
verbally report their thoughts related to each item on the survey. The cognitive lab was 
conducted individually by phone. The participants were emailed the draft survey and asked not 
to look at it until the phone call was initiated. Revisions were made on the basis of their input 
and suggestions. Finally, the instrument was submitted to OVAE for review. The instrument was 
then fin a lized to incorporate OVAE suggestions. 

Data collection. The initial activity was to explain the project to each of the five CTE State 
Directors and to request their support and participation. All the Directors were willing to 
cooperate and expressed enthusiasm for the study. They each wrote a support letter to include 
with the request for survey completion. 

NOCTI staff prepared the database of those to whom the survey would be emailed. With the 
welcome assistance of the CTE Directors of the five targeted states, they obtained the names and 
addresses of career-technical centers and comprehensive high schools in the selected states. This 
list was limited to those schools that had one or more of the four program areas chosen for the 
study. 

Next, the staff selected a random sample of the CTE centers and of a like number of 
comprehensive high schools that offer the same four CTE programs. The size of the sample 
pulled from each state was dependent on the number of CTE centers with one or more of the four 
programs chosen for the study. 

The schools’ CTE director’s email address was researched on the school website if not available 
on the state website. Telephone contact information was also gathered for use when following up 
with non-respondents. This final sample was provided to the Pennsylvania State University 
Survey Research Center (PSUSRC). 

The PSUSRC was subcontracted to administer the survey, so as to use its specialized expertise 
and also so that the administration would be seen as strictly objective and not subject to influence 
by an assessment company. Care was taken to send the survey at a time when educators might be 
most available to respond. This meant that, because the arrangements were not completed before 
some schools were scheduled for their Spring Break, the surveys were sent after schools were 
back in session. 

The surveys were distributed first by email and followed up with emails two or three times as a 
reminder with the survey URL embedded in the message for ease of access. Telephone and email 
support from NOCTI was offered so as to respond to any questions regarding the survey itself. 
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The follow-up process to all non-respondents was made first by email and then by phone calls 
from the PSUSRC. One of the principal investigators made the State Directors aware of the 
number of responses obtained from their respective state, in the hopes that they would encourage 
further participation. This was deemed preferable to NOCTI’s contacting the responding survey 
sites directly. 

The PSUSRC conveyed respondent results on a regular basis. The NOCTI staff emailed the 
respondents a thank you message and then arranged for a gift certificate sent through 
Amazon.com. 

Case Studies 

To thoroughly investigate the status of data use in schools and provide a more complete context 
for the development of an intervention, the NOCTI team conducted interviews during Spring 
2009 with directors of seven schools that were known to the team as involved in the use of data 
for instructional improvement. These are the schools that participated in four states, three of 
which happen to have been in the sample: 

• Fort Osage Career and Technology Center, Independence, MO 

• High Desert Region, Redmond, OR 

• Eastern Center for Arts and Technology, Willow Grove, PA 

• Erie County Technical School, Erie, PA 

• Reading-Muhlenberg Career and Technology Center, Reading, PA 

• Fairfax County Virginia Public Schools, Fairfax, VA 

• Virginia Beach City Public Schools, Virginia Beach, VA 

The descriptive survey protocol investigated the types of data and depth of data use by both 
secondary CTE administrators and teachers to determine how technical assessment data were 
used to improve program curriculum and identify individual and group instructional needs. 
Schools in this sample used data from a combination of core curricular tests, industry 
certifications, and job ready skill assessments; longitudinal data provided important information 
at the programmatic level. The administrators in this sample indicated that, although professional 
development is typically centered on curriculum, there were recent initiatives to help teachers 
use their data to review low-performing competency areas. One case study was the focus of an 
April 2010 NRCCTE podcast entitled NOCTI Professional Development on Assessment Data 
Use: Case Study with Aldo Jackson of Erie Co. Technical School. 

Survey Findings 

The survey data were compiled by the PSUSRC as the surveys were returned. Preliminary data 
were sent to NOCTI for analysis and interpretation as groups of responses were received. 

Schools in the sample received several email follow-up contacts, as well as a final reminder by 
phone. Incomplete surveys were counted, in general, because responders were told that response 
to all items was not required. 
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The responses to the survey were many fewer than hoped for (a total of 87), so a powerful 
analysis was not possible, and the data and subsequent conclusions and implications should be 
viewed with caution. At least one response was received from 19% of the schools sampled, with 
career centers showing a much higher response rate (31%) than comprehensive high schools 
(6%). However, the dataset has been reviewed from a descriptive standpoint (e.g., response 
frequencies), the hypotheses have been examined in as much detail as possible, and several 
potential trends related to the study hypotheses could be seen in the data set. Incidentally, the 
difficulties in collecting responses for a more powerful analysis were somewhat related to issues 
mentioned earlier in this document. 

A complete copy of the descriptive findings can be found in Appendix 3A. Of the respondents, 
the vast majority (89.6% of administrators and 92.3% of teachers) indicated that their CTE 
students took end-of-program technical assessments. The results of these assessments were used 
for a variety of purposes, the most common of which was maintaining a continuous improvement 
process, making improvements to programs in areas in which scores were weak, and reporting to 
outside bodies. Other very common uses included documenting schools and program progress, 
and helping students receive certification for the job market. In addition, the majority of the 
respondents felt that the amount of test data they received was adequate. 

A primary objective of this study was to investigate the extent to which CTE educators use 
technical assessment data to inform instructional improvement and the sources of knowledge that 
enable them to do so. A majority of the responders indicated that they do use technical 
assessment data to make instructional decisions (68.8% of administrators and 69.2% of teachers). 
Of those who did not use the data to make improvements, most expressed a belief that they 
should be using them. Some examples of instructional changes that were made by teachers based 
in part on technical assessment data included changing lesson plans to place more emphasis in 
areas in which students scored low, adding more projects and exercises in areas in which the 
group scored low, re-evaluating textbooks and other materials, and requesting additional supplies 
or equipment. With individual students, common instructional strategies after data analysis 
included providing poorly performing students with additional assistance and using student 
strengths to motivate them. 

When asked how they learned to use data to make instructional decisions, the most common 
mode among administrators was during teacher or administrative training (31.3%), followed by 
self-taught (18.8%). For teachers, the most common method was self-taught (30.8%), followed 
by teacher training (17.9%) and professional development while a teacher (15.4%). 

A second objective of this study was to examine the types of professional development CTE 
educators have received related to the use of data and how those offerings were perceived. 
Among the administrators who responded, 68.7% had received such professional development. 
Among teachers, 64.1% had received professional development. Of those who had received such 
professional development, most seemed to feel that the training contained information they 
needed and at an appropriate level. For administrators, the most common topics on which they 
had received professional development included how to interpret and apply student test data, 
information on types of tests and test items available, and the meaning of test-related technical 
terms. For teachers, the most common topics were how to measure student and classroom 
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improvement over time, the meanings of technical terms used on tests (e.g., norms, means, 
standard deviations) and information on types of tests and test items available. 

A third objective of this study was to determine how educators rated the types of professional 
development and what forms of professional development on use of assessment data they felt 
would be most useful. The results were mixed, but they did seem to indicate that a mixture of 
formal training and practical follow-up would be the most helpful. 

Respondents were also asked about areas for which they did not have professional development 
but which they wished were available. For teachers, the areas most frequently cited included 
training on what questions test data can and cannot answer, information on appropriate and 
inappropriate uses of test data, information on how tests are developed and what makes a good 
versus a poor test, the meaning of technical terms used on tests, how to interpret group-level test 
data, and how to select the most appropriate measure for the curriculum. For administrators, the 
most frequently cited training was how to interpret student-level test data; how to interpret 
group-level test data; how to compare classroom or individual data to school, district, state, or 
national averages; how to measure student and classroom improvement over time; and 
information on types of tests and test items available. 

Respondents were also asked whom they would prefer as a delivery agent for professional 
development in the use of assessment data for data-based decision making. The top three choices 
for teachers were a knowledgeable teacher or peer (preferred by 53.8%), a consultant (preferred 
by 12.8%) and a representative from a professional testing organization (preferred by 12.8%). 
The top four choices for administrators were a school, district, or state data specialist (preferred 
by 20.8%), a knowledgeable teacher or peer (preferred by 18.8%), a consultant (preferred by 
16.7%) and a representative from a professional testing organization (preferred by 16.7%) 

As indicated above, many respondents indicated that they do use data for the purpose of making 
instructional improvements, although many administrators (77.1%) also indicated that they felt 
their teachers had a need for additional training on the use of assessment data for data-based 
decision making. Overall, both teachers and administrators indicated that they had a positive 
perception of the value of technical skills assessments, although administrators had a slightly 
more positive view than did teachers (83.3% of administrators indicated that they had a “very 
positive” or “somewhat positive” perception; for teachers, 71.3% fell into those categories). 

Respondents were also asked whether their opinion on the value of technical skills assessment 
had changed over the past five years. For administrators, 50.0% indicated that it remained the 
same, as did 53.8% of the teachers. However, many indicated that their opinion had become 
more positive (37.5% of administrators and 25.6% of teachers). For those individuals, the most 
common reasons given involved the fact that they gained a greater understanding of the need for 
and application of test data, and that they had seen the value by applying it. Of the 
administrators, 4.2% indicated that their opinion had gotten more negative, as did 7.7% of 
teachers. Of those, the major reason was concern over the compromise of true student learning 
due to an overemphasis on tests. 
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The current study posed four research hypotheses, the first of which was that educators who 
know more about test data interpretation will tend to use the data for instructional improvements 
than those who know less. To examine the data statistically, a proxy knowledge variable was 
calculated by adding respondents’ reported teacher training in interpreting assessment data and 
making instructional adjustments based on assessment data (pulled from Survey Item 34) and 
professional development in how to interpret student-level test data, how to interpret group-level 
test data, and how to measure classroom improvement over time (pulled from Survey Item 40). 
This variable was significantly correlated with whether or not an educator used data to make 
instructional improvements (Survey Item 16: r = .314;p = .003). A similar correlation was 
found when comparing the knowledge variable to number of specific types of instructional 
adjustments made at the class level (Survey Item 21: r = .266; p = .013) and the individual level 
(Survey Item 23: r = .252; p = .018). However, it should be noted that the correlations reported 
here and below are small and may have little if any practical significance. 

A second research hypothesis in the study was that those who use test data for program 
improvement perceive an impact from the changes. The descriptive findings show qualitative 
support for this hypothesis, as can be seen in Survey Items 16-18 and 21-23 in Appendix 3A. 
Some of the types of instructional changes educators made are discussed above. When asked if 
they found those changes to be effective, the majority of respondents indicated that they did see 
them as effective (58.4% of administrators and 70.7% of teachers). The rest of those who 
responded to the question indicated that they were unsure. 

To examine the data statistically, perceived effect (very effective or somewhat effective) was 
correlated with whether or not an educator used data to make instructional improvements 
(Survey Item 16). No significant results were found (r = .087; p = .553). When perceived effect 
was compared to the number of specific types of instructional adjustments made at the class level 
(Survey Item 21), there was still no significant effect, although the correlation was stronger (r = 
.230; p = .112). When perceived effect was compared to the number of specific types of 
instruction adjustments made at the individual level (Survey Item 23), there was a small 
significant effect at the p < .05 level (r = .319; p = .026). Given that most respondents who 
made changes to instruction based on data reported seeing an effect, the restriction of range in 
the data is the likely cause of the lack of statistical significance. 

A third research hypothesis in the current study was that those who use test data for program 
improvement have had professional development on the topic. Various responses to several 
questions in the descriptive findings indicate that educators see the value of using data to make 
instructional improvements, and that professional development can be helpful in learning how to 
interpret and apply data most effectively. 

To examine the data statistically, a proxy professional development variable was calculated by 
adding respondents’ reported professional development regarding how to interpret student-level 
test data, how to interpret group-level test data, and how to measure classroom improvement 
over time (pulled from Survey Item 40). This variable was significantly correlated with whether 
or not an educator used data to make instructional improvements (Survey Item 16: r = .244; p = 
.023). A significant correlation was not found at the p < .05 level when relating the professional 
development variable to number of specific types of instructional adjustments made at the class 
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level (Survey Item 21:r = .181;p = .094) and the individual level (Survey Item 23: r = .185; p 
= .085). 

This current study also had a fourth hypothesis, that there would be no significant differences 
between the comprehensive high school CTE responses and the regional career-technical center 
program responses. Respondents from the two groups were compared using t tests on the 
variables used in the first three hypotheses (knowledge, receipt of professional development 
related to data interpretation, use of data to make instructional improvements, perceived effect) 
and no significant effects were found. However, the number of respondents from comprehensive 
high schools was very small ( n = 10), so no conclusions about differences can be made. 

In addition to the objectives and research hypotheses discussed above, the survey also solicited 
information on a variety of other factors that were determined to be potentially helpful in 
addressing a fourth objective, to be completed beyond Year 2 — that of creating a professional 
development system geared toward improving educator understanding of the use of technical 
assessment data and increasing the use and effectiveness of such data in making instructional 
improvements. To that end, a number of other questions were asked, including how assessment 
data are disseminated, by whom, and how frequently. Respondents were also asked about issues 
such as time available to analyze data, ability to work in teams when interpreting data, and the 
processes used to make instructional decisions. A summary of the responses to these questions 
can be found in Appendix 3A. 

Case Study Procedure and Findings 

According to the literature (Cromey, 2000; Dembosky, Pane, Barney, & Christina, 2005, 
Schmoker, 2003), many teachers and schools lack the skills necessary to make effective use of 
data; however, NOCTI project staff found through early interaction with State Directors that 
some school administrators have embraced the process to learn how to do so and have 
implemented positive changes as a result. NOCTI determined that it would be helpful to include 
a component of qualitative descriptive research by collecting and presenting the practices of 
several individual schools in a variety of states. An interview protocol was developed to elicit the 
relevant information, and individual project staff members contacted and interviewed the director 
of each school. The resulting case studies provided a context for understanding the types of data- 
driven improvements considered feasible within a school setting. 

One such site in the High Desert District of Oregon used assessment data to make curricular 
changes. In this district, end-of-year assessment data have exposed lower-than-the-norm scores 
in blueprint reading, and teachers have been tasked with developing instructional strategies to 
raise these scores. In addition, instructors at the community colleges have been involved with 
secondary teachers in designing instructional enhancements in this area. Thanks in part to an 
interest in learning analysis and review of end-of-program technical assessment data, this district 
experienced a doubling of its professional development budget. Although the site is relatively 
new (three years) to the utilization of assessment data as a path to improvement, staff and 
administration now speak a common language throughout the High Desert region. There is a 
solid understanding of the relationship between standards, assessment data, and program 
improvement. In addition to the doubling of professional development resources, collaboration 
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with community colleges has resulted in a full articulation program now in place with the 
Oregon Institute of Technology. This site has increased its credibility and has been finding more 
uses for technical assessment data each year. 

Another site, Reading Muhlenberg Career & Technology Center (RMCTC) located in Reading, 
Pennsylvania, tracks a variety of data, including technical and academic assessments and 
industry certifications, and has used these data to review program trends, adjust curriculum, and 
work to increase the number of industry certifications awarded. In addition, RMCTC now has 
two coaches to help teachers with academic integration skills in literacy and numeracy. The 
RMCTC Director indicated that the school is more data- rich and results-oriented than a few 
years ago. 

Fort Osage Career and Technical Center (FOCTC), located in Independence, a suburban city in 
Northwest Missouri, serves five school districts with 15 programs representing eight of the 
career clusters. FOCTC uses several industry certifications as well as NOCTI tests to assess the 
competencies of their program completers. Due to the level of detail provided, NOCTI pre- and 
posttests enable FOCTC to see both program and individual student growth. Analysis of 
disaggregated data and alignment with both curriculum and national standards contribute to 
instructional strategy development. These data are woven into the school culture; the pre- and 
posttest results by major area are compared with the amount of time spent on instruction in that 
area in an attempt to recognize patterns. As a result of FOCTC work in assessment, there is much 
greater focus on national standards, and the level of academic and technical instruction has 
increased. According to the Director, the analysis of the NOCTI test blueprints and comparison 
to areas of instruction was an eye-opener for some of the staff, especially in the technical areas 
FOCTC has made plans to continue to improve the quality of their programs and their 
instruction. There is a common understanding of the objectives and FOCTC leadership has 
provided both the resources and the guidance to continue their advancement. 

In May 2007, International Organization for Standardization (ISO) published updated guidelines 
to facilitate the implementation of quality management systems in education organizations 
(2007). Erie County Technical School (ECTS), located in Erie, Pennsylvania, and serving 11 
school districts, exemplifies how continuous improvement can be linked to ISO 9001 standards 
by their use of technical assessment data. Administrators and staff have collaboratively been able 
to “drill down” to find “root causes” of curricular issues impeding program improvement. There 
is a solid understanding of the relationship between standards, assessment data, and program 
improvement. ECTS teachers also maintain a program level alignment to CIP (Classification of 
Instructional Program) codes to ensure that the curriculum will reflect the nationally accepted 
content. Thanks in part to the analysis of end-of-program technical assessment data, and a team 
of 14 dedicated educators, ECTS was among the first schools in Pennsylvania to achieve the ISO 
9001 certification. 

Continuous improvement efforts benchmarked to award criteria are another spur to professional 
development that involves data collection and analysis. An example is the state-level Virginia 
Governor's Exemplary Standards Award program. The purpose of this program is to raise the 
rigor and quality of CTE programs across the state. This is a two-step process: Program 
instructors work with business advisory groups and postsecondary faculty to validate their 
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attainment of rigorous standards, and then apply for the Governor's designation. This is a 
continuous quality improvement process engaging K-12 and higher education, the business 
community, and state, regional, and local officials. The opportunity to earn this distinction 
creates an incentive for programs to meet high academic standards and improve other measures 
of program quality, strengthen their partnerships and alignment with postsecondary education 
and industry, and demonstrate relevant and positive outcomes. All CTE programs are eligible to 
seek exemplary status. The criteria for the awards ensure that all programs earning exemplary 
status will raise the science, technology, and mathematics (STEM) literacy of participating 
students through rigorous academic and programmatic standards. In determining an exemplary 
program, the evaluation criteria include program excellence, educational significance, evidence 
of effectiveness and success, and replicability and usefulness to others; these standards were 
adopted originally from the National Dissemination Center for CTE at Ohio State University. 
Exemplary programs are identified through documented nominations followed by site visits. 
Programs earning this distinction form a growing network of exemplary programs to share best 
practices with each other and with other programs striving for the designation (Virginia Career 
Education Foundation, 2009). 

One of the awardees of the Virginia Governor's Exemplary Standards Award is the Virginia 
Beach City Public Schools (VBCPS) that consists of 11 secondary schools and two Career 
Centers. VBCPS has adopted the Virginia Governor's CTE Exemplary Standards as the 
operational standards for all district programs. Teachers and administrators focus on trend data 
from end-of-program assessment as they plan for school improvement. These data support the 
effectiveness of CTE. The facts on student success, documented by assessment data, are used to 
justify equipment and resource purchases. VBCPS educators also discuss end-of-program data 
with their business and industry advisory members. Leadership at VBCPS clearly wants to use 
solid, reliable data to help their teachers succeed. According to state reports, over 96% of CTE 
students make a successful transition upon graduation to the workplace, higher education, or the 
military. 

Also in Virginia, Fairfax County Public Schools (FCPS) consists of 25 diverse secondary schools 
in which programs in all 16 clusters serve approximately 26,861 students. In addition to 
academic tests, students are assessed in “soft skill” areas; a cadre of teachers has assembled 
resources that serve as both curriculum and a professional development tool. FCPS uses a third- 
party assessment from NOCTI (Workplace Readiness), and information on competence in this 
area is collected by the state. In the area of technical skill assessment, FCPS uses 32 different 
licensures or certifications, including NOCTI assessments, to assess the majority of their 
completers. FCPS has utilized information gleaned from the assessment data for a variety of 
school improvement purposes. From a policy standpoint, the board has established a goal to 
increase the percentage of certifications gained by class each year. In addition, FCPS has used 
the data to make curricular changes - “beefing up” certain areas of the curriculum and re- 
sequencing lessons. FCPS has begun to use pretests to gauge individual progress and help 
identify gaps. This focus on continuous improvement has fostered collaborative efforts, as lead 
teachers who are more familiar with assessment have been guiding other teachers. Although new 
to the utilization of assessment data as a path to improvement, the Director says that FCPS staff 
and administration see the benefits and have begun to address related issues surrounding 
assessment and data, thereby becoming much more sophisticated in the use of data for 
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improvement of instruction. Specifically, due to FCPS’ size, consistent and well-timed 
professional development can be an issue, as can coordinated test ordering and lab space 
utilization for testing. FCPS has designed a centralized ordering process and a team to help 
disaggregate results of testing. In addition to the increasing emphasis on using these data for 
professional development, the data have helped inform the collaborative curriculum rewrites that 
FCPS participates in with business and industry partners. 

Year 1 Discussion 

Lack of skill and training in technical skill assessment, data interpretation and its use is 
especially acute among CTE teachers. Many enter the teaching field via alternative routes and 
may not receive basic training that traditionally prepared teachers receive as a part of their 
education (Bottoms & McNally, 2005). This situation was supported by the findings in the 
survey conducted in the first year of this project. In that survey, almost 46% of the administrators 
responding indicated that their teachers had not received general professional development in the 
use of technical assessment data. When asked about professional development related to specific 
uses of test data (e.g., how to interpret the data, how to apply it) between 41% and 54% of 
teachers indicated that such professional development was not available but that they wished it 
were. Of those who used data to make instructional decisions, over 30% indicated that they were 
self-taught. 

In the survey conducted by the authors in the initial phase of this project, the majority of 
respondents indicated seeing value in using standardized test data. Of those who were not using 
such data to inform instructional improvements, the majority indicated that they felt they should 
be using them. 

Importantly, even though the results reflect a relatively small sample, these survey and case 
study results provided a confirmation of the critical need for professional development on the 
topic of using technical skills assessment data to make improvements in instruction. In addition, 
the findings provided specifics about what the nature of that professional development should be. 
What the literature reveals about the type and characteristics of professional development 
reported as state-of-the-art for effectiveness was corroborated by the survey. 

The findings from the survey research and case studies were used to create new professional 
development opportunities that were designed and piloted in the following year of the project 
These were designed to be cost-effective and efficient and to include strategies and approaches 
that: 

• Will improve and increase instructional personnel’s knowledge, skills, and ability to 
help students meet challenging and rigorous academic and CTE skill proficiencies; 

• Will advance instructional personnel’s understanding of effective instructional 
strategies that are by scientifically based research; and 

• Are sustained, intensive, and classroom-focused. 



46 




A more detailed description of the professional development design and pilot phase is found in 
the following section of this chapter. 



Year 2: Development and Piloting of the Professional Development Intervention 
Purpose and Goal of Year 2 Activities 

The overall purpose of the Year 2 activities was to use (a) the results of the Year 1 research on 
educator use of assessment data and (b) guidance from the literature for the creation of a 
professional development program geared toward secondary CTE teachers and administrators on 
how to effectively interpret assessment data and use the information to make instructional 
improvements in the classroom. The developed intervention was to include the materials 
necessary for implementing the training, a facilitator/delivery system for maintaining the system 
within the school setting, and a preliminary pilot test evaluation of the effectiveness and 
feasibility of the program. 

The goal was to develop and pilot professional development for CTE educators on the use of 
technical assessment data for data-driven decision making that is both based on sound principles 
of learning and delivered in a manner that is sustainable within a school system. For the cycle to 
be complete, it is imperative that educators be able to diagnose problems by looking at evidence 
(data), receive the training to be able to plan instructional modifications around that evidence 
(Butler & McMunn, 2006), and then be motivated to apply the training in the classroom. 

Three research questions formed the basis of the professional development for K-12 educators 
targeted toward the use of assessment results: 

1. Have educators increased their knowledge about technical assessment data as a result 
of the professional development intervention? 

2. Are educators able to apply their knowledge of technical assessment data to improve 
instruction as a result of the professional development intervention? 

3. Will educators be more motivated to apply their learning about technical assessment 
data to instructional improvements as a result of the professional development 
intervention? 

The intervention was to be a professional development strategy that, although research-based, 
could be provided in a very practical manner. Facilitators selected for their ability to engage 
participants in the topic provided highly interactive, project-based initial training. Unique aspects 
of this intervention are the opportunity for participants to work with data from their own students 
(via technical skills pretests) and to work with a cohort of peers to develop action plans that 
relate to their specific schools and classrooms (Wayman & Cho, 2008). After the initial training, 
participants were coached over several months in their school environment by the same 
facilitators who provided the training. Educators were also encouraged to interact with others 
who were receiving the professional development, thus forming a professional learning 
community, which has been shown to be helpful for learning and growth through collaboration 
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(Ball & Cohen, 1999; Borko, 2004; Stone et al., 2008). Principals and administrators were 
included in the professional development to help provide critical support within the workplace 
environment (Dembosky et al., 2005; Mason, 2002). Because much of the training and 
development occurred on school grounds, the financial costs to the school were minimal. 

Further, because many of the activities took place in the context of work that is already being 
done by educators, the professional development was highly contextualized, guided by learner 
needs, and easily implemented and maintained within the school environment. 

The following sections provide detailed information about how this intervention was developed 
and pilot tested. 

Methodology 

In addition to the survey and case study findings from Year 1 of this project, the findings yielded 
in the literature review also informed the creation of the pilot intervention. Other NRCCTE 
professional development projects were also consulted to provide additional information. One 
such source was the SREB modular materials for alternative certification of CTE teachers, which 
are discussed in Chapter 2. The NRCCTE Math-in-CTE project also yielded proven principles 
for the professional development (Stone et al., 2008) 

In preparing to develop the intervention, a framework of criteria was established to guide the 
work. An understanding of the underlying processes related to the use of assessment data for 
improving instructional and student outcomes was sought from the survey analyses, as well as 
the malleable factors to be addressed. Contributions from the literature provided guidance on 
distinguishing between effective and less effective professional development characteristics. The 
findings from the initial survey as well as additional contributions from the literature review and 
other NRCCTE projects informed the development and refinement of the professional 
development strategy in terms of content, structure, delivery, and maintenance. The literature 
review continued for the purpose of enhancing the professional development materials and 
process as the project continued. The case study results provided an additional resource for 
development of the intervention. 

Selection of the Sample 

Although the term assessment data can be very broadly defined, the primary focus of this study 
was on standardized summative assessments. In addition, although several forms of data were 
addressed, the main focus was on technical assessment, relevant because it is now required by 
Perkins IV. Furthermore, although many educators can benefit from professional development 
on the use of assessment data, the target audience (or population) was defined as teachers and 
administrators of secondary CTE programs, as they were the subjects of the Year 2 study and 
those whose needs were researched. The researchers assumed that the selection of these 
educators, who have much in common and must all deal with the same focal point of technical 
assessment, would make it easier for the educators to comprehend the material and apply it to 
their schools and classrooms. 
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A total of nine secondary CTE school sites in five states were to be selected to pilot the 
professional development on the use of assessment data. Five of these were intended to be in 
Round 1 of the pilot and four in Round 2, which was planned to start a month later so as to allow 
for the application of learning from Round 1. Selection criteria were based in part on survey 
findings and in part on phone interviews with the administrator and a pilot site visit. The first 
priority was to select sites from among those who participated in the previous year’s survey. 
Schools seen as most suitable included those with (a) multiple CTE programs from among 
Business Education, Construction, Health Science, and Manufacturing, (b) an interest in 
improving their use of data for decision making, and (c) willingness to participate in the training, 
coaching, and data collection phases of this project. For each of the five states, schools meeting 
these criteria were culled from the survey responses and prioritized on the aforementioned 
criteria and with regard to a distribution among urban, suburban, and rural sites. These were then 
discussed with the CTE State Director in each state to get input on suitability and other factors 
operating within the state. 

For most of the states, this process was sufficient to move forward with calls to the potential sites 
to explain the project and seek agreement to participate. In Illinois, however, few sites met the 
criteria, partly because of the relatively small number of survey responses. It had been agreed 
that, in such a case, other states and schools would be evaluated for participation. In consultation 
with the State Director of CTE, the decision was made to work with two career academies in the 
Chicago Public Schools (CPS), and their agreement to participate was gained within the CPS 
system. Because this would possibly bring the total number of pilot sites to nine, permission was 
sought and granted by the NRCCTE to move forward with the understanding that the budget 
implications would not be problematic. 

Project staff developed a site call protocol to ensure that all relevant information, including both 
the responsibilities and benefits of participation, would be conveyed to the administrator of each 
of the potential pilot sites. A single member of the project staff was designated to make the calls 
to gain some standardization of approach. In a few cases, the school administration decided, after 
consideration, not to participate. 

As a result of these calls, sites that responded positively were visited (a) to gain a more thorough 
understanding of their intentions and potential to be responsive to the initiative and (b) to offer a 
more complete description of the project and details related to participation directly to the sites’ 
educator teams. Project staff members were assigned to visit schools in specific states. They used 
a site visit handout and a team-developed protocol to ensure that no details were inadvertently 
left out. Subsequent to the visit, they shared their notes regarding the visit with other project 
staff. These same staff members continued to monitor the progress of the pilot test in the states to 
which they were assigned. One exception to this process was in Oklahoma, where the two sites 
were not brought on board before the Facilitator’s Training, so those visits were made 
subsequently by the facilitator. 
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The list of pilot sites is as follows: 

• Illinois: Prosser Career Academy and Simeon Career Academy, Chicago, IL 

• Missouri: Rolla Technical Center, Rolla, MO; South Central Career Center, West 
Plains, MO 

• Oklahoma: Central Tech Career Center, Drumright, OK; Kiamichi Tech, McAlester, 
OK 

• Pennsylvania: Clearfield County Career and Technology Center, Clearfield, PA; 
Swenson Arts and Technology High School, Philadelphia, PA 

• Virginia: Valley Vocational Technical Center, Fishersville, VA 

The administrator and a total of two to five teachers from each school participated in the program 
so as to encourage communication among the group and the development of a community of 
practice. In one case, two administrators participated, whereas in another, no administrator 
participated, due primarily to a misunderstanding about the requirement for an administrator to 
be part of the team. In some cases, a test coordinator joined the team. The total sample size from 
all sites numbered 48 individual educators. 

Developing a research-based intervention. A summary of the literature conducted by Smith et 
al. (2003) indicated that the effectiveness of different types of professional development is 
subject to debate, and that philosophies of how professional development should be delivered 
have shifted from more traditional models, such as workshops, toward such models as study 
circles, coaching, collaborative problem-solving groups, and practitioner inquiry. Although 
specific lists differ somewhat, research indicates that an ideal effective professional development 
for K-12 teachers should (a) be of longer duration, (b) make a strong connection between the 
development topics and the context of their work, (c) include a strong emphasis on analysis and 
reflection, (d) include a variety of activities, (e) encourage teachers from the same workplace to 
participate together, (f) be continuous and ongoing, and (g) be connected to comprehensive 
change processes focused on improving student learning (Banilower, Boyd, Pasley, & Weiss, 
2006; Darling-Hammond et al., 2009; Dembosky et al., 2005; Guskey, 2003; Lewis, 2000; Smith 
et al., 2003; Sparks, 1994). 

Unfortunately, many forms of teacher professional development still fall short of this ideal 
(Darling-Hammond et al., 2009) and are generally focused on short-term, standardized sessions 
that usually do not result in improved content knowledge, altered teaching practice, or enhanced 
student learning (Kahle, 1999). Efforts were made to find the most effective and feasible way to 
provide useful professional development for today’s CTE teachers. 

Given the increasing use of scores on high-stakes achievement tests, meaningful professional 
development specifically geared toward the use of assessment data must have a strong work 
connection, link to a comprehensive change processes, have strong teacher involvement (Klein, 
Hamilton, McCaffrey, & Stecher, 2000) and also address concerns and misunderstandings about 
the nature of tests and test data among parents, students, teachers, and administrators alike (e.g., 
Charles, 2008; Cizek, 2005; Herman & Golan, 1991; Phelps, 2009). Unfortunately, the available 
research also indicates that most teachers and administrators do not have the skills necessary to 
make effective use of data (Cromey, 2000; Dembosky et al., 2005, Schmoker, 2003). 
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It is logical to assume that many of the same factors influencing the effectiveness of teacher 
professional development elsewhere will also be important for professional development on the 
use of test data and that such factors would apply to CTE teachers. However, it is also important 
to note that CTE teachers may have some different requirements (de Moura Castro, 2009). CTE 
teachers are expected to meet their students' needs for career development, technical and 
academic achievement, and information technology skills. Therefore, CTE teachers must 
interpret data from technical skills assessments related to their program area and be cognizant of 
the student’s grade level in language arts and math. It can also be argued that CTE teachers serve 
a more diverse student clientele (NRCCTE, 2010). Finally, the rapidly changing workplace and 
innovations in technology require ongoing technical upgrades (including evolving theories, new 
procedures, new materials, industry standards revisions, and new equipment) and curriculum 
content revisions (NRCCTE, 2010). These differences would indicate that, although the content 
of the training provided may vary, CTE teachers’ professional development needs and the format 
for maximum retention and application should be similar to those of other teachers. 

As indicated above, there is virtually no literature regarding professional development for CTE 
educators in the use of data to make optimum program and instructional decisions, thereby 
improving student outcomes. This research-based intervention begins filling this gap by focusing 
on developing an intervention for CTE that places data for decision making through the use of 
assessment results within a CTE-specific professional development framework, yet uses the 
principles identified for effective professional development for all educators. In October 2009, 
the National Governors Association Center for Best Practices (NGA Center) issued a brief 
(Grossman & Hirsch, 2009) that outlined approaches states can take to improve the quality of 
teacher professional development for the ultimate purpose of aiding student learning. Among the 
suggestions designed to develop new models of professional development, one is especially 
relevant for this effort, namely the use of student learning data to create individualized 
professional development plans for teachers. John Thomasian, director of the NGA Center said, 
in the press release for this brief (Munley, 2009); "As a part of this, it is important to refocus 
teacher professional development to ensure that it has real-world implications for classrooms and 
a positive impact on student performance” (p. 1). This intervention uses both achievement and 
occupational skill data from the participating educators’ students to customize the professional 
development; the thrust of the participating educators’ work is to develop and implement an 
action plan for improving classroom instruction. 

Structure of the Intervention 

Following the recommendations of Grossman and Hirsch (2009) and the model of change 
presented at the beginning of this chapter, the intervention involves multiple stages, each 
discussed in more detail below. Several of the stages overlapped in time and included (a) the 
development and refinement of the professional development training, including content matter, 
structure of the training (e.g., layout, exercises), delivery, and related facilitator materials, (b) the 
selection and development of measurements to assess the effectiveness of the learning and the 
successful application of that learning to the classroom, (c) the selection of appropriate pilot 
sites, (d) the selection of facilitators/coaches, (e) the administration of the training and coaching, 
and (f) the initial evaluation of the program process and effects. 
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The Materials Development Process 

The initial materials developed for the intervention were embedded in the Facilitators’ Guide, 
which contained all educator worksheets and handouts and a PowerPoint presentation for the 
initial training workshop. The Guide contained four major components: (a) content related 
specifically to the study and how it is being conducted (e.g., what types of interactions they are 
expected to have with the educators and how often; what kind of data will be gathered and 
when); (b) content that the educators will receive; (c) content directed specifically toward the 
delivery of the training (e.g., interactive activities, strategies for differentiation to allow 
individual educators to address the specific needs of their students and programs); and (d) 
content related to establishing and maintaining a facilitating-coaching-guiding relationship with 
the participants as they apply learning to their school settings, as well as evaluation forms. The 
educators participating in the pilot workshops received copies of the presentation slides and the 
worksheets and activities for each of the five steps in the model. 

The intervention, CTEDDI (Career and Technical Educators using a Data-Driven Instruction 
model), presents a five-step model, shown in Figure 3.2, that is intended to be the basis for an 
annual data cycle used by educators. 

Brief background material on how standardized tests are developed, different common formats 
of standardized tests, and similarities and differences from formative and locally developed 
assessments were embedded in the content of the training. The content also included a summary 
of the types of information usually reported on standardized test reports (including at the group 
and individual level), what common terms mean and how they are related (e.g., average, mean, 
percent), and how they should be interpreted in an applied setting. Exercises were incorporated 
on external factors that can impact individual test scores, as well as group test scores and trends 
over time. In addition to appropriate uses of test data, inappropriate uses were reviewed. 
Participants learned not only how to interpret different levels and types of assessment results, but 
also strategies for using those results as diagnostic tools that enable a focus on instructional 
strengths and weaknesses, making adjustments, and tracking the impact of those adjustments. 
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Figure 3.2. The five-step data cycle of CTEDDI. 

Evaluation Measure Development 

Several instruments were used to measure and evaluate the results of the professional 
development strategy that focused on using assessment results for improving instruction with the 
purpose of improving student outcomes. The initial measure was a questionnaire to gather 
information about prevailing practices prior to training, including how each of the pilot schools 
had used technical assessment data in previous years. This information was used to provide a 
baseline for comparison with the educator’s practice in response to the intervention. 

The next measurement is a content pre- and posttest designed by the project staff and reviewed 
and revised by additional subject matter experts. This assessment served as a knowledge measure 
to assess participants’ learning and retention of using data for decision making. It was closely 
aligned to the content standards used to develop the training and measured such factors as 
educator knowledge of data terms, understanding of how to interpret assessment data, and 
knowledge of how to apply such interpretations to making instructional improvements. The 
assessment served as a measure to determine that participants did, in fact, gain knowledge from 
the initial training. Educators responded to objective items, rather than simply self-reporting on 
their perceptions of learning to the research staff. This instrument was administered at three 
points: (a) prior to receiving the training, (b) immediately after receiving the training, and (c) 
near the conclusion of the project. The second and third assessments included an added 
performance component to the knowledge assessment, which included an evaluation of the 
action plans for data use that participants develop during the training and the subsequent 
implementation of those plans in their school environment. Through the engagement of 
participants, use of technological tools and resources, and application of in-depth understanding, 
it is possible to assess educators’ s ki ll in interpreting complex, real-data situations and 
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determining how to use data to improve their educational programs and student instruction. 
Longer-term retention was measured by the third administration of the assessment. 

Midway in the process, participants received a questionnaire asking them to report their 
perceptions about their use of data, their motivations to use data, the impacts on student 
achievement they witnessed or expected to witness based on the use of data, the impact of the 
facilitation process on their use of data, and success stories or challenges encountered. The main 
purposes of these measures were to assess implementation of the action plans developed during 
the training, assess the transfer of training to other aspects of instructional improvement (e.g., 
what other ways are educators using instructional data in addition to the implementation of the 
initial action plans), assess the motivation to continue using data to make instructional 
improvements in the future, and track the events and circumstances that assist or hinder 
educators in applying their improved knowledge and skills in data use in an actual school 
environment (e.g., time availability, willingness/ability to work collaboratively, coach 
interactions, support for changes). A similar questionnaire was administered at the end of the 
project. Facilitators were also surveyed about the process based on their experiences with the 
participants, and they participated together in discussing their reactions on a final group 
conference call. 

Facilitator Selection and Training 

To provide the training at these pilot sites, a total of six facilitators were selected, one located in 
each of the participating states and in one state, two who elected to share one position. The 
facilitators were selected based on their skill and experience with training and informal coaching; 
represented were former CTE state directors, former local superintendents, a specialist at a state 
professional development center, and professional development consultants. All needed to be 
willing and available to participate in all phases of the project (including travel to initial training, 
school visitations, etc.). 

The Facilitator Training was planned for collaborative involvement of the Principal Investigators 
and the primary materials development specialist in working with the six facilitators. Plans were 
made to train the facilitators on the material, delivery methods, and subsequent follow-up 
strategies in a face-to-face meeting, with provision for input from the facilitators on the materials 
and delivery plan. 

The facilitators were invited to the training well in advance and scheduled so that all facilitators 
were able to attend and interact. The training was held in a location in relative proximity to the 
NOCTI Main Office, so that materials could be transported easily. The facility’s training room 
was selected with regard for the conversational nature and physical set-up of the room with a 
large round conference table and superior electronic connectivity. 

Project staff trained the facilitators as planned. Once the facilitators were trained in the process, 
they made contact with each of their schools in their state and scheduled their introductory visits 
and the one-day intervention workshop. They continued to provide the ongoing coaching and 
mentoring program to the selected educators from each of their schools. Additional follow-ups 
with and among facilitators occurred via web conferences as the project progressed, so that the 
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program could be iteratively improved and the facilitators could interact on the basis of their 
experience in implementing it. 

The Initial Implementation 

The CTEDDI intervention was piloted at nine sites in five states. There were two rounds of pilot 
testing during the spring semester of 2009. The professional development materials and process 
were revised significantly after each round of the pilot process; as the initial workshops were 
held at pilot sites, both facilitators and educators contributed to the improvements, with the most 
extensive revisions made between the two workshop rounds. The project staff regards this 
iterative process as one of the strengths of the project. 

The formal professional development workshop is devoted to interactive activities, such as case 
studies that demonstrate analysis of relevant datasets, and developing a plan to apply the training 
in the educators’ own settings. This training was designed to incorporate standard data analysis 
and interpretation techniques that would be nationally generalizable. 

In the professional development pilot process, educators were encouraged to use data relevant to 
their school situation. To have a basis for the initial instructional improvement plans, students in 
the teachers’ classes took a standardized technical assessment at two points — once as a pretest to 
provide material for use in the initial professional development training and again as a posttest to 
provide an additional opportunity to practice their skills on real student data. Thus, these tests 
were a source of data for educators to use both for exercises and application. Educators from 
each pilot school developed a plan as a part of the training and implemented it at their local 
schools with facilitator coaching. Teachers were interested to compare the data from the pre- and 
posttests for themselves; however, the tests were not used to draw conclusions about student 
achievement, which was beyond the scope of this project. 

The use of standardized data from their own students not only provided a link to their classrooms 
to make the training more relevant, but also provided teachers with the information needed to 
customize and contextualize their initial action plans to the needs of their particular students, 
programs, and schools (e.g., providing remediation to particular students, adding more teaching 
time for particular subject matter, using more exercises and hand-on activities, providing more 
resources for teaching particular material). Schools that were currently using technical 
assessments could choose to use those existing data for the professional development process. 

For those who did not use technical skills assessments, NOCTI donated from its standard test 
battery the assessment and subsequent data report that most closely matched the school’s 
program. The facilitators assisted those pilot sites needing assistance to obtain student data from 
NOCTI. 

NOCTI tests, both at the professional level and at the student level, have been a standard of CTE 
programs for almost 45 years; these assessments are current, valid, and reliable reflections of the 
needs of industry in over 70 different occupational areas. Any lack of alignment between test and 
curriculum content was taken into account when the data were interpreted (e.g., building trades 
students taking carpentry tests), as were other relevant factors (e.g., depth of content as 
determined by hours of participation). It was important to stress that educators should have 
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relevant data to use in the professional development process; however, these data did not have to 
be from the same test source. 

In summary, during the intervention development and pilot phase, educators had standardized 
data relevant to their particular schools or classes to use during the professional development 
experience. It is generally assumed that teacher knowledge and classroom practice mediate 
between professional development and student achievement. However, measuring the indirect 
effect of teacher preparation on student achievement is imprecise, so this was not attempted in 
this time period, given that the primary purpose of the study was to impact the data-driven 
decisions of CTE educators, thereby improving instruction. 

Initial Action Plans 

The main goals for this research-based intervention were to have the educators produce an 
implementable action plan based on an analysis of their program’s data and to be able to use the 
data cycle annually for continuous improvement. Reports from the facilitators and drafts from 
educators at the pilot sites provided an overview of the types of problems derived from data that 
were transformed into action plans. At the conclusion of the workshop, the educators 
summarized their findings on an action planning form that specified the goals, indicated the 
overall strengths and weaknesses in student performance reflected in the data, and listed the 
high-priority goals with resources, steps, strategies, data required, and indicator(s) of success. 
Work on the action plan occurred after the workshop and was accomplished with mentoring from 
the state’s facilitator. 

Educators entered the planning process from many different perspectives, which were reflected 
in their plans. Carpentry teachers perceived the need to ensure that standard terminology used in 
carpentry and the construction trades was being properly incorporated into lessons. Another 
construction trades teacher observed a weakness in his students’ knowledge of interior finishing 
based on his program’s data. A business teacher noted weaknesses in the students’ knowledge in 
areas of accounting, computation, and record keeping. A health sciences instructor conducted a 
gap analysis and identified two test sections on which to focus her planned interventions. Two 
other health sciences teachers engaged in discussion over the professional development sharing 
website on strategies being used to teach specific concepts. One teacher planned to strengthen 
study skills with her students. Overall, the teachers agreed that this planning process was a 
valuable exercise. 

Some administrators intuitively assisted their staff with the planning process, providing feedback 
and resources. One administrator focused on attendance issues; however, others removed 
themselves from the process. Based on the varied participation of the administrators, their role is 
being better defined as the workshop intervention guide is revised yet again in keeping with the 
iterative process. 

The Iterative Process 

Information concerning various aspects of the intervention was sought through various channels 
that included internal and external reviewers, the facilitators, and the educators at the pilot sites. 
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The initial package of intervention materials included the Facilitators’ Guide, the related series of 
exercises and worksheets, and a PowerPoint presentation. After multiple internal reviews and 
revisions, the materials were reviewed externally before being used at the pilot sites. A review 
response form was developed to encourage some standardization in the form of the responses 
(see Appendix 3B). The participants were sent information by email about the purpose of the 
study, its methodology, and the objectives of and process for the type of review to which they 
had agreed. 

Reviewers were drawn from several relevant groups and recruited to assist with materials 
development: 

• content experts in standardized testing and data analysis and presentation (two 
people) 

• subject matter experts in training development and delivery (two people) 

• subject matter experts in beginning and fostering small learning communities or 
“communities of practice” (two people) 

• educators who are familiar with using data from standardized assessments (two 
people) 

• educators who are not familiar with using data from standardized settings (one 
person) 

After receiving detailed reviews from these experts, the materials were revised on the basis of 
their input and suggestions. Next, the facilitators were able to review the materials to suggest 
improvements. 

After the pilot program was delivered, a systematic process was established to make iterative 
changes and improvements based on information gathered in the field from participants and 
facilitators. These revisions were then incorporated into the fully developed intervention. At the 
conclusion of Round 2, the evaluation data were reviewed and several structural changes were 
made to the intervention materials. Specifically, the following were added to the materials: more 
stringent criteria for workshop participants, a better defined role for the school administrator, 
more structured mentoring in the post- workshop phase, tools for monitoring student learning, 
and a series of post-workshop activities using an online professional development community. 

After the school year ended at all nine pilot sites, the NOCTI project staff conducted a web- 
based conference call with the facilitators to discuss their overall perceptions about 
improvements that could be made for the next year, as well as to review the changes noted 
above. Further refinements to the product and process will be made based on upcoming reviews 
of the latest revisions, with the intent of producing a complete program of research-based 
technical assistance for CTE schools wishing to implement an annual data cycle. 

The Post-Workshop Mentoring Phase 

To meet the need for sustained support after the full-day intervention and to achieve the coaching 
and collaborative problem-solving environment advocated by research (Banilower et al., 2006; 
Darling-Hammond et al., 2009; Dembosky et al., 2005; Grossman & Hirsch, 2009; Guskey, 
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2003; Lewis, 2000; Smith et al., 2003; Sparks, 1994), the facilitators remained with their pilot 
sites for several months after the workshop. The pilot sites participated in several structured 
mentoring sessions conducted by their facilitator. The groups met on a regular basis to discuss 
their data plans, problems encountered, and issues that arose. 

A CTEDDI online professional development sharing site was developed to meet the need for 
interaction and support during the implementation process (see evaluation data in Appendix 3C). 
The website was developed exclusively for the pilot participants and their facilitators. An 
account was created for each workshop participant and each educator was required to log on and 
post at least one message at that time. Because educators were located in five states, they had 
never met face to face, thus presenting a challenge to their communications. A review of the 
educators’ interactions revealed that only a few of them engaged in substantial exchanges on this 
website, primarily to discuss instructional texts and curricular resources. In light of the 
interactions of the majority of the educators, which did not advance beyond the introductory 
level, combined with evidence on the formal evaluations, it was evident that pedagogical 
components must be incorporated into this online community. 

Structuring the Online Community 

The main design elements for online learning communities are the social and pedagogical 
components (Ryman, Vine, & Richardson, 2009), which provide the basis for revisions to the 
website. To provide the pedagogical structure for the CTEDDI online community, a series of 
authentic learning tasks in the post-workshop interval will be implemented with structured 
facilitation. Facilitators, asked to provide an initial review of these additional activities, felt that 
the activities were an appropriate addition to the intervention. Post-workshop activities are 
intended to increase collaborative interactions and higher-level thinking among the educators; 
they will also help with the challenging social aspects of building an online community of 
practice that is distributed over different locations. 

The five main functions of a community of practice are building relationships, sharing, learning, 
knowledge creation, and collaboration (iCohere, Inc., 2010). Additionally, Wenger, McDermott, 
and Snyder (2002) stated that characteristics of such a community should be excitement (e.g., 
novelty), relevance, and value. Incorporating these factors into the design of the website 
increases the probability that the community will sustain itself over time by eliciting substantial 
interactions, thus avoiding becoming little more than a network of friends. The main community- 
building strategy for CTEDDI will be to require community members to participate in varied 
events to accomplish the five main functions. The first community activities will once again be 
introductory, needed to build relationships (Ryman et al., 2009), but as part of the post-pilot 
revisions, the educators will now complete a series of authentic learning tasks, such as posting a 
revised lesson plan for review by other educators who respond with suggested strategies or 
resources; such a design will engage the educators in sharing and collaboration through relevant 
tasks that have immediate value to the educators. 

The role of the facilitators is not diminished in the post-workshop phase, as they have the 
challenge of promoting and sustaining constructive social dynamics and learning dialogues in a 
distributed community. Morrissey (2000) noted that the creation of a professional learning 
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community does not occur quickly or spontaneously, but is the product of an intentional effort on 
the part of an administrator or facilitator. Membership in a learning community involves a 
paradigm shift from short-duration workshops to a sustained professional learning experience 
intended to bring about an overall change in the way teachers view the teaching process. 
Therefore, facilitators must manage the community’s interrelated social and learning aspects by 
maintaining an appropriate flow of discussion in an online learning environment to achieve the 
critical thinking and problem solving necessary to transition to a cyclic data-driven process of 
instructional improvement and curricular planning (Ryman et al., 2009). 

Deeper learning through participation is a concept that is becoming prominent in professional 
development practice. By embedding learning tasks in the post-workshop interval, the 
anticipated result will be collaboration, higher-level learning, and an overall change in the 
instructional process. Community activities, such as sharing products and ideas, bond members 
through shared practice, which is no stranger to teachers, because empowering students to 
participate in shared learning is the basis of a teacher’s work. Through participation in a 
structured community of practice with prompts from a facilitator, teachers can reflect on their 
classroom practice and better understand how they influence learning. As Smith (2009) 
summarized, learning lies in relationships between people, and there is an intimate connection 
between knowledge formation and activity. 

Results 

A full summary of the data gathered from participants can be found in Appendix 3C. However, 
selected findings are summarized below. 

Pre-workshop questionnaire. The pre-workshop questionnaire was distributed by and returned 
to the facilitators, so it had a 100% response rate. Participants were told by the facilitator and in 
the questionnaire that they did not have to answer any questions they did not want to answer. 

In the data gathered prior to the workshop, most of the participants (89.1%) indicated that their 
schools did use some form of end-of-program technical skills assessment, even if only a teacher- 
designed classroom assessment. Most participants indicated that their schools used end-of- 
program assessments for multiple purposes. For administrators, the top three purposes, endorsed 
by the most participants, were (a) to maintain a continuous improvement process (85.7%), (b) to 
make improvements to programs in areas in which scores are weak (85.7%) and (c) to help 
document school and program progress (78.6%). Among teachers, the top three purposes were 
(a) to maintain a continuous improvement process (70.4%), (b) to make improvements to 
programs in areas in which scores are weak (55.6%), and (c) to help students receive certification 
for the job market (5 1.9%). 

Although most participants indicated that their schools used end-of-program technical 
assessment data, many also indicated that they themselves could use some improvement in using 
data skillfully, with teachers reporting a greater need than administrators. Figure 3.3 below 
shows the breakdown of self-reported skill level. 
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Figure 3.3. Average rating of the degree of skill respondents personally felt they have in using 
technical assessment data for classroom improvement. 

On the knowledge assessment administered prior to the workshop, administrators did score 
slightly higher than teachers (68.6% compared to 60.9%). The pretest scores varied substantially 
from site to site, ranging from a low of 44.4% to a high of 72.2%. 

Post-workshop questionnaire. At the end of the workshop, participants completed another 
questionnaire that gathered feedback on the perceived quality of the workshop content and 
effectiveness of facilitators, the perceived impact of the professional development on knowledge 
and skills related to use of standardized end-of-program technical skills data, and ways in which 
participants felt the workshop would be useful when applied in their schools. Participants also 
completed the knowledge assessment again. This questionnaire was distributed by and returned 
to the facilitator. One initial participant self-selected out of the project about one half-hour after 
beginning; not including that person, the post-workshop questionnaire had a 100% response rate. 

In terms of overall quality of the workshop content and materials participants generally provided 
high ratings, with an overall average rating of 5.20 on a six-point scale. In general, teachers gave 
slightly higher ratings than administrators. When asked to rate various aspects of facilitator 
effectiveness, participants were also very positive, providing an average overall rating of 5.34 on 
a six-point scale, with teachers again providing slightly higher ratings. When asked to rate the 
ways in which participants expected the professional development would impact their 
organization, the top three, based on overall ratings, were (a) “how I plan instruction,” (b) 
“student learning outcomes,” and (c) “the way my organization uses data.” 

On the knowledge assessment, a modest gain of about 8% was seen overall, with administrators 
and teachers showing similar gains. In general, the knowledge assessment contained two distinct 
types of items. The first type measured knowledge of assessment terms such as mean and 
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standard deviation , whereas the second type involved interpreting data provided in charts and 
tables and drawing conclusions about student and classroom needs based on those data. Overall, 
a gain was seen in the assessment terms items (a gain of 20% from pre- to post- workshop), 
whereas there was little change in the interpretation items. This may indicate that participants 
came in with a greater ability to interpret data than knowledge of the technical terms related to 
assessment. 

Because the nature of this intervention was to make iterative improvements to product and 
process, many of the items included on the post- workshop questionnaire were open-ended and 
intended to solicit comments and suggestions for improvements; a great deal of attention was 
paid to the comments. 

When asked what changes they anticipated as a result of the workshop, 6 administrators and 28 
teachers provided comments. Administrators and teachers most commonly indicated that that 
they would look more closely at data and use data more in making decisions. Several 
administrators also planned to work more closely as a team within their schools when analyzing 
data. Many teachers indicated an intention to alter their lesson plans based on data (e.g., 
extending time, changing focus, altering classroom activities). Several teachers also mentioned 
an intention to review more or re-teach areas in which students were struggling and to alter 
teaching strategies where it seemed appropriate. 

Participants were also asked for feedback on what they felt the key strengths were. For this 
section, 13 administrators and 26 teachers provided comments. Most commonly mentioned by 
both administrators and teachers was the inclusion of actual data on their current students as a 
part of the project. Teachers also highlighted gaining a better understanding of data and being 
able to set goals specific to their classes. 

When asked for feedback on key weaknesses, 13 administrators and 21 teachers provided 
comments. The two most common comments were that more time was needed for the workshop 
and that the materials needed to be organized better (this comment was mostly from the first- 
round participants, and changes were made to the materials for the second round). Other 
weaknesses that were highlighted included the need for more activity in the workshop (e.g., long 
day of sitting, needs to be more exciting), that the workshop should take place earlier in the year, 
that more activities should be built in for administrators (most were teacher-focused), and that 
more discussion should be included on actual strategies for addressing student weaknesses seen 
in the data. Several administrators and teachers indicated that they saw no areas of weakness. 

When asked for specific suggestions for improvement, 14 administrators and 16 teachers 
provided comments. Several indicated they saw no need for improvement, but of those who 
made suggestions, better organization of content and materials was highlighted, along with more 
time for the workshop and more focus on reviewing the school’s actual data and discussing 
strategies and goals. 

One final question in the post- workshop survey asked participants if they would recommend the 
workshop to others, and why or why not. Of the 15 administrators that responded, 14 indicated 
that they would and one indicated that he/she would not. The reason this person gave was the 
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lack of activities geared toward administrators. Of the 26 teachers that responded, 25 indicated 
that they would. The one who indicated that he/she would not indicated that it “still remains to be 
seen how this will impact the shops.” Based on this feedback, it appears that, overall, participants 
were pleased with the content and process of the workshop. 

Some evidence of the promise of the intervention for achieving the intended outcomes can be 
noted from verbatim feedback received from the facilitators after their workshops: 

• “It went well and was well received by the teachers. I found that there was an overall 
understanding of the importance of using data but a lot of uncertainty about how to 
use the data. The teachers are anxious to really analyze their own data. We need to 
make sure we encourage them to use as much data as they have at first to really get to 
root causes of the achievement gaps.” 

• “The workshop was a great success and the feedback from the participants was 
extremely positive.” 

Additional evidence emerged from the educators themselves, as they posted comments on the 
sharing site to be discussed in the next section. One example is the following: “I am excited my 
program is a part of this project. I think the information I have already acquired from the pretest 
scores has given me valuable insight as to curriculum changes I need to make. This project will 
help me to see the changes that need to be made to better prepare my students for the world of 
work or higher education. Thank you for the opportunity to grow professionally and improve my 
program.” 

Interim questionnaire. Midway through the mentoring portion of the project, participants were 
mailed another questionnaire to gather feedback about the ongoing mentoring as well as any 
additional recommendations for the workshop they may have had in retrospect. Participants were 
also asked about their activities and progress on their goals at that point. Because this 
questionnaire was mailed to participants and returned directly to the data analyst, the response 
rate was lower, even after several reminders. However, the response rate was respectable, at 
87%. 

Participants were asked to provide their opinions about the mentoring/coaching portion of the 
project as well. In general, participants were still positive about the professional development 
and indicated that they were using it in their schools and making progress on their action plans. 
They also indicated that the professional development was having an impact on such tasks as 
how they planned instruction (an overall rating of 4.41 on a six-point scale), and how they 
monitored student progress (an overall rating of 5.53 on a six -point scale). 

The participants indicated that they were comfortable working with their facilitators (an overall 
rating of 5.37 on a six -point scale), and that the mentoring was helpful in supporting continued 
learning and in implementing the action plan (overall ratings of 4.68 and 4.71 on a six -point 
scale). In general, they were satisfied with the amount of contact they had with their facilitators 
and with the methods of contact. Differences from state to state in some of the responses (e.g., 
the number and nature of contacts pursued by the facilitator) provided valuable information in 
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how to select and train facilitators in the future, as well as how to better structure the facilitation 
process, and this information will be incorporated in the next round of revisions. 

When asked for suggestions for improvements to the mentoring portion of the project, 6 
administrators and 11 teachers provided comments. The participants expressed a desire for more 
structure in the mentoring. More cross-school communication and more face-to-face interaction 
with the facilitators were also frequent themes, as was more structured guidance from the 
facilitators. Another frequent theme was the need to start the professional development earlier in 
the year to allow for more mentoring time and more time to enact the action plans. When asked 
about difficulties they had encountered in implementing the action plan, the most common 
constraints mentioned were lack of time to work on the action plan and lack of time to meet as a 
group. 

Participants were also asked about successes they had encountered. Six administrators and 21 
teachers provided comments. Several indicated that they had seen positive improvements based 
on the instructional changes they had made. Such instructional changes included reviewing areas 
of general weakness, finding new materials and resources to use with the students, adding to the 
curriculum or changing curriculum timing, and assisting or getting assistance for individual 
students to address weaknesses. Several also commented that knowing there was a study going 
on and seeing their pretest data seemed to motivate their students. 

When asked about roadblocks, 8 administrators and 26 teachers provided comments. By far the 
most frequently mentioned roadblock was time — time to schedule the tests, time to implement 
the action plan, time to meet with facilitators, and time to cover all the material. Other 
roadblocks mentioned included the need for more student data and problems with having to re- 
teach material that the students had not retained. 

The interim questionnaire also asked participants for any recommendations for improvement to 
the initial workshop. Eight administrators and 22 teachers responded. Suggestions included 
making the workshop longer, building in more time to work on action plans and goals, starting 
earlier in the year, providing more direction for what to do after the workshop, having more 
student data available, and ensuring that facilitators were sufficiently knowledgeable about the 
topic. 

When asked about improvements to the facilitation process, 10 administrators and 21 teachers 
provided comments. The most common suggestion was more structure to the mentoring and 
more communication. 

Although they saw areas that needed improvement, overall the participants saw value in the 
project, and indicated that they saw value in using technical assessment data for making 
instructional improvements. Many indicated that they believed they would continue to use 
assessment data for making such improvements after the project was over and that participating 
in the project had increased their desire and willingness to do so. 

Suggestions from facilitators for improvement in the mentoring/coaching portion of the project 
include the development of a workbook that has structured tasks for each of the follow-up 
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sessions. Examples suggested were that schools within a state could be asked to discuss an issue, 
or the whole group could discuss an article they had read in common. Follow-up questions could 
be discussed to test retention of workshop content. Another suggestion was to include a process, 
form, and examples for tracking additional non-assessment data such as attendance and 
demographic information. Overall, more structure in the mentoring activities would be of 
perceived value to facilitators as well as to participants. 

Final questionnaire. At the end of the project timeline, the participants were given one final 
questionnaire. This questionnaire, like the pretest and posttest questionnaires, was distributed by 
and returned to the facilitator in person at the final group meeting. In one case, an individual was 
unable to attend the final meeting so was given the questionnaire to complete and mail in. This 
participant did not return the survey, so the response rate for the final survey was 98% rather 
than 100%. 

Overall, the participants indicated that they perceived their skill in using data had increased due 
to the professional development program (a mean rating of 4.72 on a six-point scale, as are all of 
the following means), that the skills had been applied in their classrooms (4.88) and that the 
program was helpful in planning for improved instruction (4.93). The highest ratings for the 
areas impacted were in planning instruction (4.70) and monitoring student progress (4.60). The 
participants were generally comfortable working with their facilitators (5.52), felt they had made 
adequate progress on their action plans (4.86), and felt that the other members of their team were 
working together effectively (4.93). Overall, the participants indicated that they saw technical 
assessment data as a valuable tool for making instructional improvements (5.22), and that they 
would continue to use data after the project was concluded (5.34) 

As with the other questionnaires, the final questionnaire focused on open-ended questions related 
to suggested improvements and the strengths and weaknesses of the program. When asked about 
improvements to the facilitation process, 8 administrators and 16 teachers provided comments. 
The most common suggestions were more time for the mentoring, more contact with facilitators, 
and more structure to the mentoring portion of the project. Most felt the work they had done on 
their action plans was a success, but felt that more time was necessary to fully see the results. 

When asked about roadblocks, 10 administrators and 24 teachers provided comments. By far the 
most mentioned roadblock was insufficient time to work on the project plans. When asked about 
successes, 10 administrators and 23 teachers provided comments. The most mentioned successes 
included perceived improvements based on spending more time on student weakness and 
changes to the curriculum. Increased student scores on the posttests were also highlighted. 

The last component of the final questionnaire was a re-administration of the knowledge 
component. The overall mean score dropped somewhat from the posttest (67.6% compared to the 
posttest mean of 71.9%), but did not drop to the level of the pretest (63.7%). When the two types 
of knowledge items (knowledge of assessment terms and data interpretation) were examined 
separately, the drop could be seen to be due mostly to a drop in scores in the knowledge of terms 
items (a loss of 12%) whereas there remained little change in the interpretation items. Figure 3.4 
shows the shift from pretest to posttest to final. 
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Figure 3.4. Pretest, posttest, and final percent correct on items related to terms and interpretation. 

The gain and then subsequent drop in scores related to knowledge of terms may indicate that 
participants came in with a lower knowledge level and thus were able to gain more knowledge in 
the workshop. The subsequent drop may indicate a lack of retention that could be alleviated with 
more review and exercises related to terms built into the mentoring process. Although the above 
results may indicate that participants are coming in with suitable knowledge in data 
interpretation or are not gaining additional knowledge through the program, there may still be 
gains in ability to apply the knowledge to their particular situations and make targeted plans to 
address findings. This was not a part of the knowledge assessment. 

However, it should be noted that, when site averages were examined separately, the 
interpretation of the situation was less clear cut. As can be seen in Figure 3.5, the same trend of 
pre-post gain, post-final loss for items related to terms was relatively consistent across sites. 
However, the situation with the items related to interpretation show more variance across sites. It 
is likely that other factors (e.g., amount of incoming knowledge, facilitator workshop delivery 
quality, facilitator mentoring quality, site-specific factors) are having an impact. 
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Figure 3.5 Pretest, posttest, and final mean percent correct by site for items related to terms and 
interpretations. 



Discussion of Year 2 



It was recognized that, in order for this intervention to operate as intended, it needed to be 
effectively implemented on several levels: 



• First, the initial training had to be developed and delivered in a manner that improved 
participant knowledge of the use of data, provided them with the tools to transfer that 
knowledge to the school environment (such as through the development of an initial 
action plan), and enhanced their motivation to apply the knowledge to make 
instructional improvements. 
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• Second, participants needed to be willing and able to actively apply their knowledge, 
skills, and plans to their school environment in ways that were (a) relevant to the 
needs of their particular schools, programs, and students, (b) feasible given their 
particular circumstances, and (c) intended to and likely to result in instructional 
improvements that might lead to gains in student achievement. 

• Third, the professional development had to be an ongoing process that began with the 
initial training but continued in the school setting via facilitator coaching and 
participant collaboration. The professional development also needed to actively 
address barriers to implementation as they arose. 

• Fourth, the intervention had to show measureable gains in participant understanding 
and use of data in a school setting, and instructional improvements based on the use 
of data. 

• Finally, the participants needed to show a motivation and intention to continue to 
apply their skills in the future. 

With each of these conditions met, the intervention did succeed and can be expected to result in 
lasting, integrated changes in the use of standardized assessment data to inform instructional 
improvements and would also be likely to result in increases in student achievement. The 
findings resulting from this project will continue to contribute to improvement of practice in 
CTE. It seems essential that if technical assessments are to be taken by students for the purpose 
of measuring achievement, the resulting data should be used, not only for accountability needs, 
but also to assist educators in improving programs and individual instruction to encourage higher 
achievement. 

Professional development to be offered in future years, as a result of having credible findings on 
which to base them, should be of benefit in assisting the field. It is anticipated that states will be 
likely to request such offerings, and that it will be possible to establish a train-the-trainer model 
to spread knowledge widely. 



Discussion 

The Year 2 pilot yielded a great deal of information on ways to improve the intervention as well 
as the intervention’s areas of strength. Participants experienced a number of successes based on 
the intervention. Such successes included spending more time on areas in which students were 
weak and changes to the curricula based on test results. Many participants indicated that they 
saw individual and classroom improvements based on the changes made. Although student test 
scores were not a variable in the study, many teachers indicated seeing improvements at the 
posttest. Participants were also asked about barriers they encountered in the process, and 
overwhelmingly the most common barrier mentioned was lack of time - the compressed timeline 
for the project, as well as a lack of time to work on action plans. 

Throughout the pilot, opinions and ideas for improvement were gathered from both participants 
and facilitators. Based on the feedback received, a number of additional changes and 
improvements are planned for the process and materials. These include, but are not limited to: 



67 




• Altering the timeline of the program so that the initial workshop occurs early in the 
school year and the mentoring time is increased. 

• Possibly lengthening the initial workshop or moving certain elements into more 
structured mentoring activities. 

• Incorporating more exercises into the workshop specifically for administrators. 

• Developing more structure for the mentoring portion of the process, including a 
calendar with suggested timelines for facilitators, structured exercises and activities 
for facilitators to use with teams during the mentoring process. 

• More scheduled meetings and contacts between facilitators and teams. 

• More suggestions for inter-site activities and discussions for facilitators to employ 
where appropriate. 

• More structured activities to encourage inter-school conversations via the sharing site. 

In the following year, more review and revision took place using the pilot site participants from 
the current study as well as teachers from other programs in the pilot sites plus educators from 
other schools and states. The facilitators from the initial pilot study have played an ongoing role 
in continuing to refine the materials and process. In order to take the intervention to scale, it was 
important to first broaden the scope of the reviews that inform refinements. Although the 
intervention was iteratively improved during the year, some necessary adjustments in the design 
were of a variety that were not suitable for revision in midstream, such as allowing more time by 
starting earlier in the school year. Changes such as these were made to allow the intervention to 
function more as intended. It was also desirable to further review the refined intervention to 
ensure that it would be deemed likely to function similarly in additional clusters, with a broader 
variety of assessments, in various school types, and in a larger pool of states. Reviews of the 
improved materials by educators in new sites in the pilot states and new sites in new states and 
also by subject matter experts were conducted, thus using an alternate but in-depth process to 
verify the effectiveness of changes. Further iterative improvements were made as a result of 
these reviews. 

Data-driven decision making in education is here to stay. NCLB is based on accessing and 
utilizing student achievement data. NCLB and Perkins IV both rely heavily on professional 
development to achieve their goals. Perkins IV specifies that standards-based technical 
assessments in CTE be administered and that each state implementation plan must describe how 
professional development will, among other charges, assist in accessing and utilizing data 
including occupational information, student achievement data, and data from assessments. If the 
spirit and not just the letter of both Perkins and NCLB are to be adhered to, it is critical that 
student achievement data are not simply gathered and reported, but also used to inform 
instruction and make classroom-based improvements that should ultimately lead to higher 
student achievement. These favorable outcomes depend on educators receiving effective 
professional development to acquire skills in using and interpreting data from standardized 
technical assessments. 

This intervention addresses both those needs by providing educators with (a) the knowledge and 
skills they need to understand and use assessment data for instructional improvements in a 
manner that meets standards for effective professional development, (b) the tools and resources 
to apply those skills in their school settings, and (c) the coaching and motivation to work 
collaboratively to continue to use their skills in a focused and integrated manner. As educators 
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integrate data use into determining, making, and evaluating the effectiveness of instructional 
improvements, these improvements will be more effectively targeted toward the specific needs 
of their students, programs, and schools, resulting in higher quality improvements and a more 
focused use of resources. As instructional improvements become more targeted and effective, it 
is anticipated that student achievement is likely to improve, resulting in better prepared students 
entering higher education and the workforce and subsequently in long-term gains in workforce 
quality, productivity, and global competitiveness, goals important not only to the CTE field, but 
to the nation as a whole. 

The fully refined professional development model developed by this project — CTEDDI (Career 
and Technical Educators Using a Data-Driven Improvement Model) — will be made available to 
states in NRCCTE Year 5 (2011-2012). CTEDDI will provide educators from participating states 
with professional development intended to increase their knowledge and skills in the use and 
interpretation of assessment data for the purpose of making instructional improvements. The 
professional development will be delivered by facilitators who will also serve as coaches for the 
educators for applying their initial training at their school sites. 
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Chapter 4 



Where We Need to Go 

It has been over a quarter-century since A Nation at Risk (National Commission on Excellence in 
Education, 1983) was published, but the pressure for educational reform that it started has not 
abated. The calls for higher standards and increased accountability have, if anything, become 
more pervasive and powerful. The Common Core State Standards, which appear likely to be 
adopted by almost all states, and the Race to the Top competition conducted by the U.S. 
Department of Education (2009), have increased the emphasis on test scores as the dominant 
measure of educational quality. CTE has been and will continue to be impacted by these 
pressures. As Kazis (2005) put it: 

There is no realistic way to significantly improve high school outcomes without tackling 
secondary career and technical education. CTE is too significant a segment of the high 
school enterprise, and its traditional role as a track [f]or the “non-college bound” means 
that a large proportion of the students who need the most support to achieve at high levels 
are enrolled in CTE programs or schools. The future of high school reform cannot be 
addressed without including vocational high schools and programs within comprehensive 
schools, (p. 4) 



The Expanded Mission of CTE 

The associations that represent CTE educators have embraced the need for an expanded mission, 
one that includes an increased emphasis on academics and preparation for postsecondary 
education. The NASDCTEc advocates at the national level on behalf of the officials responsible 
for how CTE operates in the states. The association recently published Reflect, Transform, Lead 
(NASDCTEc, 2010b), which states: 

CTE reflects the modem workplace. And since the majority of careers require a 
postsecondary credential, high-quality CTE programs incorporate rigorous academic and 
technical standards, as well as critical workplace skills such as problem solving, 
communication and teamwork, to ensure career and college success for its students, (p. 1) 

To further advance the direction in which CTE is moving, NASDCTEc joined with ACTE, the 
largest association representing CTE educators, and the Partnership for 21 st Century Skills to 
publish Up to the Challenge (2010). This report highlights the ways in which CTE content and 
pedagogy complement the Framework for 21 st Century Learning developed by the Partnership 
and urges the incorporation of the Framework and CTE throughout education. The linkage with 
the Partnership is a concrete example of how the leadership for CTE is attempting to make the 
field a key part of high school reform, as has also been advocated by Brand (2003), Kazis (2005), 
and the National Governors Association Center for Best Practices (2007). 

The University Council for Workforce and Human Resource Education (UCWHRE) represents 
most of the universities that offer four- year teacher education programs and award doctoral 
degrees in CTE. In 2005, UCWHRE published a book that examined the need for leadership in 



70 




CTE. One of the chapters in that book (Camp & Johnson, 2005) addressed the need for a broader 
theoretical framework for CTE that specifically includes the reinforcement of academics through 
contextualized applications. 

The increased pressure on CTE to improve the academic skills of its students stems from a broad 
consensus that, as a society and an economy, we can no longer accept a large proportion of our 
young people leaving education without a sound academic base. In an economy characterized by 
rapid rates of change, workers must be able to continually learn and adapt. For those students 
who choose CTE, the contextualized application of academics provides an opportunity to learn 
skills that have not been acquired in previous classes. Cross (1992) identified the “pedagogic 
pluses” of what was then called vocational education as: (a) student involvement, (b) feedback 
between teacher and student due to the observable, easily assessable nature of the tasks taught, 
and (c) high standards based on external certification and advisory committees that provide an 
outside check on student performance. The National Research Council and the Institute of 
Medicine (2004) have summarized the research documenting how occupational content can be 
used to increase student engagement and motivation to learn. 

But the challenge is enormous. For most of its history, CTE served the non-college bound by 
preparing them for employment upon leaving high school (Kazis, 2005). To ensure that the skills 
taught in the classroom were the same as those used on the job, CTE teachers were (and still are) 
required to have several years of work experience in the occupational areas they teach. There is 
no guarantee, however, that work experience teaches the full range of skills in any given 
occupation. NOCTI was originally established to determine if such workers actually knew and 
could perform the skills they were applying to teach. This requirement of being able to perform 
is an important qualifier because written assessments of occupational knowledge and skills have 
only modest correlations with assessments of actual performance (Nuttall, 1987). Nevertheless, 
the knowledge and skills of expert workers has remained the starting point for the development 
of occupational curricula. Instead of relying on individual teachers, information for curriculum 
development is usually gathered from workers by structured methods, such as surveys, job and 
task analysis, and the guided group discussion method DACUM. an acronym for Designing A 
CUrriculuM (Norton & Moser, 2008). 

The occupational grounding of the CTE curriculum must remain paramount if students are to be 
prepared for employment, but technical learning must be supplemented with increased attention 
to the academic theories and constructs that support and reinforce that technical learning. 
Increasing the impact of participation in CTE courses on academic performance represents one 
of the main challenges facing the field. One drawback of the work experience requirement for 
CTE teachers is that the academic underpinnings of CTE teachers who enter teaching from their 
occupations are not like those of teachers who follow the traditional teacher preparation path. 
Those in the technical arena are focused on task completion, whereas those in education are 
focused on the process of learning and acquiring information. The trajectory of most regular 
secondary teachers includes high school completion, college completion, and teaching. The 
trajectory of many CTE teachers includes high school completion, work experience, and then 
entry into teaching. If secondary CTE is to carry out its expanded mission, ways must be found 
to assist CTE teachers to fulfill the expectations for the teaching of both technical and academic 
content that have been placed upon them. 
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Responding to the Challenge 



The two NRCCTE projects discussed in this report address some of the professional 
development implications of these expanded expectations for CTE. The NOCTI project 
addressed the technical side by developing the CTEDDI professional development model 
(Career and Technical Educators Using a Data-Driven Improvement Model). CTEDDI is 
designed to increase the ability of CTE teachers and administrators to use data from technical 
skill assessments to improve program planning and instruction. The materials and procedures 
developed by NOCTI are based on an extensive review of the relevant literature and survey 
research that determined what teachers and administrators knew and wanted to learn. Initial 
drafts of the materials were field tested and the feedback from these tests was used to revise the 
model. The revised model has undergone additional review and iterative refinement, and 
planning has begun for a national roll-out of the model in 201 1-2012. 

CTE teachers are familiar with teaching to industry standards. Many programs have state or 
national licensing and certification bodies that set standards that program completers must meet. 
Table 2C.9 in Appendix 2C presents an extensive list of organizations that conduct certification 
and summative testing of occupational skills. The training provided by CTEDDI will enable CTE 
educators to use data from such testing to identify skills and concepts needing increased attention 
in their classes in order to bring all students to the level that they can pass the technical skills 
tests used in their occupational areas. 

Using test results in this way is one of the core components of continuous quality improvement 
and strongly encouraged by the accountability requirements in Perkins IV. Martinez (2007) 
included accountability principles as part of the foundations upon which all CTE programs 
should be based. As the field has implemented these requirements, it has found many “broken” 
systems; among them, the inability to effectively track students from high school to 
postsecondary education to work; to report assessment or certification data consistently; and to 
benchmark acceptable standards (Kotamraju, 2010). Field testing of the CTEDDI model has 
helped to point out and correct flaws in local data collection systems. All indications from the 
U.S. Department of Education, such as the criteria for Race to the Top, imply that this emphasis 
on accountability will continue when the federal CTE legislation is reauthorized in Perkins V. 
CTEDDI can help local districts provide a solid response to the Perkins IV requirements for 
program improvement and upgrading of teachers’ instructional abilities. It also can enhance data 
reporting and collection (via the Consolidated Annual Reports that all states must file) and will 
assist in providing evidence of longitudinal gains that indicate that CTE, in general, is having a 
positive effect. 

The SREB project addresses the pedagogic skills needed by alternatively certified teachers and 
places a special emphasis on the integration of academic and technical instruction. Like the 
NOCTI model, the SREB induction model is research-based. The four concept papers in the 
Appendices to Chapter 2 summarize the research upon which the modules used in the induction 
model were based. These modules were iteratively field tested, and a test of the full model began 
with training that was conducted in the summer of 2010. The 10 teachers who participated in the 
summer training are receiving mentoring, coaching, and administrative support and are taking 
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part in an electronic community of practice during the 2010-2011 school year. Information 
obtained from this full-year test will be used for additional refinements of the model. The model 
emerging from these refinements will be experimentally tested during the 201 1-2012 school year 
by randomly assigning some alternatively certified teachers to participate in the model and others 
to serve as a control group. 

When the SREB model has been fully tested and revised, states will have available a 
scientifically verified approach to providing alternative certification. The evidence that has been 
reviewed in this report implies that the need for alternative certification, which is already high, 
will only increase. NASDCTEc (2010a) published a four-part analysis of trends in CTE based on 
data collected from a survey of all state directors. One of these analyses focused on the shortage 
of CTE teachers in the 16 career cluster areas. The areas with the largest current shortages at the 
secondary level are in STEM (Science, Technology, Engineering, and Mathematics); Health; 
Manufacturing; Agriculture, Food, and Natural Resources; and Architecture and Construction. 
The publication also cited two model programs for addressing these shortages: Both involve 
alternative certification of experienced workers recruited from business and industry. 

The field tests conducted by both NOCTI and SREB replicated two findings often noted in the 
research on professional development: 

• Teachers want learning experiences that they can directly apply in their classrooms. The 
NOCTI field tests provided teachers with test results from their own students, and this 
enhanced the usefulness of the training they received. The participants in the SREB tests 
explicitly requested more examples from their own content areas in the training materials, 
and the materials were revised to provide these for each participant. 

• Teachers benefit from opportunities to interact with each other to reflect on and digest 
information that has been presented. Professional development is often thought of as 
something “done to” teachers, but the best professional learning occurs when teachers 
work together to improve their own lessons and methods (Darling-Hammond et al., 

2009). The primary association of those involved in professional development has 
recently changed its name from the National Staff Development Council to Learning 
Forward to reflect a change in conceptualizing professional development from one in 
which teachers are recipients to one in which teachers are collaborators. The case studies 
developed by NOCTI on the use of assessment data to revise instruction illustrate the 
type of collaborative professional development that produces change in program planning 
and classroom instruction. 

Our experience in these projects and our assessment of the relevant literature lead us to conclude 
that providing time for teachers to work together represents a critical prerequisite for the type of 
professional development that will be needed if CTE is to significantly increase the rigor of its 
offerings. If technical rigor is to be improved, teachers and administrators need time to learn and 
apply the techniques of using technical skills assessment data to improve instruction. If academic 
rigor is to be improved, ways must be found for CTE teachers to work with their academic 
counterparts to identify and explicitly teach the academics inherent in CTE curricula. 
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Unfortunately, at a time when teachers need to collaborate more than ever, it appears that they 
have less time to do so. A recent study by the National Staff Development Council (one of the 
last it published before changing its name to Learning Forward) implied that the amount of time 
teachers have to work together has been declining (Wei, Darling-Hammond, & Adamson, 2010). 
This study analyzed data from the 2000, 2004, and 2008 Schools and Staffing Surveys, 
conducted by NCES with nationally representative samples of schools. The 2008 survey 
indicated that teachers had participated in fewer professional development experiences of more 
than eight hours than they had in 2004. In the 2000 survey, 34% of the teachers agreed or 
strongly agreed that there was “a great deal of cooperative effort among staff members;” in 2004 
this figure dropped to 17%, and by 2008 it was only 16%. 

Time for teachers to work together to improve instruction is a relatively new concept in 
American education and differs from the prevailing image of “real” teaching: the autonomous 
teacher presiding in the classroom. This image stands in sharp contrast to practices in high- 
performing Asian and European countries in which teachers spend more time in lesson planning 
and collaboration with colleagues than in direct instruction (Darling-Hammond et al., 2009). 
Providing more time for collaboration has major financial implications and may be the primary 
constraint on wider adoption of this practice. A review of current practices in financing 
professional development concluded that most districts have little information on what 
professional development costs or the results it produces (Sawchuk, 2010). 

As the economic effects of the recession that began in 2008 continue, resource constraints on 
state and local educational agencies are likely to make it even more difficult to provide the time 
needed for effective professional development. The funds that are available must be used 
effectively, and this implies fewer, more highly targeted, and longer professional development 
experiences. CTEDDI and the SREB induction models are being carefully developed with 
extensive feedback from field tests and will result in approaches that are likely to maximize the 
return on the time invested in them. 
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Appendix 2A 



Concept Paper for Instructional Planning Module 

Four modules provide the curriculum for the professional development experiences designed for 
the alternative certification induction model for CTE teachers. The purpose of this paper is to 
frame the content of the instructional planning module and describe literature and research that 
support that area of competence. Scientifically based research that informs what CTE teachers 
need to know and be able to do in planning instruction is rare. Consequently, much of the 
literature described in this paper is consistent with policy that drives CTE or with research and 
literature that reflect what is generally accepted as good practice. 

“What do beginning CTE teachers need to know and be able to do to plan instruction 
effectively?” It is widely accepted that accomplished CTE teachers plan instruction in ways that 
help students master technical and academic knowledge and skills (National Board for 
Professional Teaching Standards, 1997). Figure 2A.1 illustrates a framework for the essential 
concepts that teachers need to know and be able to do plan instruction that leads to students 
college and career readiness. The first concept is the specific nature of the content that teachers 
need to understand to plan well. The second concept is the needs of students, including their 
talents, interests, aspirations, characteristics, learning styles, and learning needs. The third 
concept is the function and use of instructional planning tools such as unit plans, lesson plans, 
curriculum maps, and course syllabi. Finally, this section will address two related areas of 
competence for instructional planning: reflective practice and the skill of working with others to 
plan effectively. 

Teachers Need to Know Their Content 

A deep understanding of content and the concepts, principles, and skills within that content is 
critical for teaching. The Interstate New Teacher Assessment and Support Consortium Model 
Standards for Beginning Teachers (INTASC, 1992) state that a beginning teacher needs to 
“understand the central concepts, tools of inquiry, and structures of the discipline(s) he or she 
teaches and create learning experiences that make these aspects of subject matter meaningful for 
students” (p. 14). As Shulman (1987) noted, “we expect teachers to understand what they teach, 
and when possible, to understand it in several ways” (p. 14). 

CTE teachers must have a sufficient understanding of the career field to guide student learning. 
Recent policy developments in the field of CTE and a shifting emphasis on the role of CTE in 
public schools have increased the depth and breadth of that content. Accomplished career and 
technical educators command a core body of knowledge about the world of work in general and 
the skills and processes that cut across industries, industry-specific knowledge, and a base of 
general academic knowledge. 
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Figure 2A.1. Essential concepts in instructional planning. 
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CTE teachers draw on this knowledge to establish curricular goals, design instruction, facilitate 
student learning, and assess student progress (National Board for Professional Teaching 
Standards, 1997, p. 13). Historically, CTE teachers needed only know the specific skills of the 
occupation for which they were training their students, but the National Board for Professional 
Teaching Standards (NBPTS) suggests that a broader understanding of work in general and skills 
that cross industries is needed. Beginning teachers need an overall understanding of this content 
and how it has evolved. 



95 















The new mission of CTE. These are times of significant change for CTE, largely due to major 
shifts in the demands of the American workforce. Eight of the 10 fastest growing occupations 
require some form of postsecondary education and the U.S. Department of Labor (2008) has 
indicated that this trend is likely to continue. About 72% of the increase in postsecondary 
education requirements is a result of higher skills demanded by employers for jobs that 
previously did not require any college (Camevale & Desrochers, 2003). Furthermore, only 40% 
of manufacturers believe that high school graduates are prepared for entry-level positions 
(National Association of Manufacturers, 2005). A survey of human resource officials (U.S. 
Department of Labor, 2001) found that 81% rated recent hires as deficient in written 
communications and 72% found recent hires to lack basic writing skills such as grammar and 
spelling. The central question seems to be whether the workforce will have the required skill sets 
demanded by the 21 st century workplace. Given these new requirements, traditional CTE 
programs that prepare students for entry-level jobs are no longer viable. 

In addition to meeting the needs of the workplace, CTE is called upon to contribute to student 
achievement and high school completion. Nearly every high school student takes one CTE 
course and some students earn more credits in CTE than they do in mathematics or science 
(Silverberg et al., 2004). High school students who combine a substantial academic curriculum 
with several vocational courses do better than students who omit either one of these two 
components (Stern, 1999). The risk of dropping out for students who entered high school at a 
normal age was also found to decrease as they added CTE courses to their curriculum until they 
reached a ratio of taking one CTE course for every two academic courses (Plank, DeLuca, & 
Estacion, 2005). CTE students who graduate with technical skills also experience a labor market 
advantage (National Assessment of Vocational Education, 2004). CTE clearly plays a role in 
engaging students, supporting them in graduating, and linking them to their futures. 

A major theme in the literature on CTE is the need for a strong link between academics and CTE 
studies, in part because there is little difference between the level of academic knowledge and 
skills necessary to enter the workplace and the level needed to go to college (American Diploma 
Project, 2004). Perkins IV, the federal law that reauthorizes funding for CTE, requires schools to 
develop challenging academic and technical standards for CTE programs and to provide 
challenging, integrated instruction. “When CTE courses also incorporate more academic rigor, 
research shows that student achievement significantly increases. These findings suggest that CTE 
should be an important aspect of a state’s broader high school redesign strategy” (National 
Governors Association, 2007, p. 1). 

ACTE (2006) has suggested that the clear goal for CTE should be career and college readiness 
for all students. In a report issued by Jobs for the Future and the Aspen Institute (2005), Medrich 
called for CTE to combine academics with career-focused studies, weed out less compelling 
programs and weak courses, and focus on the fundamentals, including engaging curriculum and 
instruction and quality assessments. As beginning teachers enter the field and engage in their 
first year of instructional planning, a sense of the changing needs of the workplace, the demands 
of federal policies and school improvement efforts, and the role of CTE in the “big picture” of 
education is essential. Beginning CTE teachers need to understand how the new mission for 
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CTE — to prepare students for both the workplace and further education beyond high school — 
will impact the content they teach in their particular CTE programs. 

Career clusters, pathways, and programs of study. The new mission for CTE implies the need 
for a broad framework of content for CTE courses. To provide a broad foundation of opportunity 
and to prepare high school graduates for further learning and the workplace, CTE content is best 
organized around occupational career clusters that include competencies necessary for a wide 
range of jobs within each cluster (NASDCTEc, 2006). Within each career cluster, there are 
opportunities to develop multiple career pathway programs of study (POS) that describe CTE 
and academic courses in a seamless series of learning experiences from Grade 9 through the 
postsecondary level. In these pathways, the CTE courses at the 9th- and lOth-grade level focus 
on transferable work skills, academics in context, and career exploration experiences. Grades 11 
and 12 include the technical core of the career pathway, including the technical skills within the 
cluster, challenging academics, and work-based learning. Postsecondary pathway experiences 
include advanced technical skills, advanced academics, and worksite experiences (Hull, 2005). 
POS sequences lead to readiness for postsecondary and the workplace at the end of high school, 
including certification and industry-recognized credentials if appropriate, and provide a seamless 
transition from high school to postsecondary studies. Because students will likely change jobs 
many times during their work careers, the concept of a pathway within a career cluster provides 
students with a broad base of transferable knowledge and skills that crosses industries. 

The States’ Career Cluster Initiative (NASDCTEc, 2006) provides an in-depth, online resource 
for CTE teachers to use in instructional planning. Each cluster is outlined with cluster-level 
knowledge and skills and sample indicators of performance. Within each cluster, there are a 
varying number of career pathways, each with a knowledge and skills chart that describes what 
workers need to know and be able to do to demonstrate competence within a career pathway. 
Depending on the state in which the beginning teacher is employed, there may be state online 
resources that have adapted the career cluster framework or link the specific state CTE 
competencies identified for each program back to the State’s Career Cluster Initiative model. 

Industry-specific knowledge and skills. Because they often enter teaching from a career in 
business and industry, beginning CTE teachers are often most familiar with the knowledge and 
skills students need to master to be successful in a particular industry. Teaching industry-specific 
knowledge and skills requires a depth and breadth of understanding that usually is obtained from 
employment experiences, a professional school, or clinical internship. It involves knowing the 
workplace culture and expectations, applicable industry standards needed for certification or a 
credential within the field, and the kinds of industry assessments and employer exams used to 
measure the skills of the field. Industry- specific knowledge also encompasses trends, new 
findings, and technology. 

Most states list industry- specific knowledge and skills to be taught in CTE programs in 
secondary CTE standards that specify what students are to master in each program. Although 
there is variability in the design of these standards from state to sstate, almost all states have 
either completed or are in the process of completing CTE standards (Castellano, Harrison, & 
Schneider, 2007). Similar to efforts to identify academic standards, CTE standards are the first 
step states take in creating a curriculum and assessment framework on which a system of 



97 




accountability is based. Because the Goals 2000 Act of 1994 funded the development of industry 
standards and Perkins IV requires local education agencies to use state-developed indicators to 
assess both CTE and academic performance, some states’ CTE standards provide information on 
how the standards align to the state academic standards or national industry standards. The first 
step a beginning CTE teacher can take in outlining course content is to locate and study the 
knowledge and skills outlined in these state CTE standards. 

Nationally developed industry skills standards, although they may be reflected in the state CTE 
standards described above, may also prove a good content resource for CTE teachers. Some 
industry associations serve as certifying bodies and publish the specific skills that students need 
to pass certification exams and earn a credential that is recognized throughout the field. The 
National Institute for Metalworking Skills (NIMS) is but one example. NIMS developed 
precision manufacturing skill standards and competency assessments and it accredits programs 
that meet its quality requirements. Over 6,000 U.S. companies recognize the NIMS standards, 
credentials, and accreditation process. As beginning CTE teachers develop their understanding 
about the content they are to teach, examining the certification exams may prove helpful. Some 
states have even begun to use these exams to assess students’ mastery of complex material that 
measures both academic and technical content (SREB, 2009). An awareness of national industry 
standards, the certification or credentialing process for individuals or programs, and the exams 
that measure performance are important for beginning teachers to consider as they make 
instructional plans. 

Academic knowledge and skills. With the emphasis on academic skills outlined in the previous 
section on the new mission of CTE, beginning CTE teachers need to be familiar with the 
academic knowledge and skills students need to graduate from high school and be college- and 
career-ready. High school CTE programs are expected to provide a relevant context for applying 
academic knowledge and skills, thereby increasing student motivation to learn challenging 
content and persist to graduation and further learning beyond high school (Bottoms & Young, 
2008). 

The first step in identifying the academic knowledge and skills that need to be taught in CTE 
programs is to check the degree to which existing state CTE standards have referenced related 
state academic standards. Some states have completely aligned their CTE standards and 
curriculum resources to state academic standards. Similarly, industry skill standards may also 
include references to academic standards. In the States’ Career Cluster Initiative mentioned 
earlier, academic knowledge and skills are integrated into the model so that each career cluster 
pathway includes English, mathematics, and science content related to the career field. 

SREB established college and career readiness indicators in English (Murray & Bottoms, 2008) 
and mathematics (Southern Regional Education Board, 2010) that CTE teachers can use to focus 
on key knowledge and skills needed to make the transition to further learning and the workplace. 
The development of the indicators involved expert panels of curriculum and national test 
developers and teachers, an analysis of major curriculum documents include the ACT College 
Readiness Standards and national content standards, and the identification of a set of indicators 
by the expert panel. Sample learning activities and assessments were developed for each 

3 See https://www.nims-skills.org/web/nims/home. 
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indicator. In order to meet the demands of the new mission of CTE, teachers will need 
professional development in indicators such as these and assistance in how to use them to create 
intellectually challenging assignments and assessments that integrate academic and CTE content 
with the goal of college readiness. 

Workplace readiness skills. There is a growing understanding and delineation of a common core 
of general knowledge about the workplace and employability skills that all employers expect of 
their employees to be successful (Partnership for 21 st Skills, 2009). These include work ethics, 
interpersonal relationship and communication skills, and leadership and management skills. The 
career cluster knowledge and skills frameworks of the States’ Career Cluster Initiative include 
these skills. Beginning teachers need an understanding of how these skills are applied in the 
workplace. 

Another emerging framework for workplace readiness skills are the 21 st century skills 
(Partnership for 21 st Century Skills, 2009), developed in an attempt to focus K-12 educators on 
the most important skills student needs to learn to meet the demands of modem life. Working 
with educators, employers, and government officials, a set of standards were identified within 
several categories. Life and career skills focus on competence related to flexibility, initiative, 
social skills, productivity, and leadership. Learning and innovation skills include creativity, 
critical thinking, communication, and collaboration. Information, media, and technology skills 
focus on information and media literacy. The standards also outline core subjects and 
interdisciplinary themes of civic literacy, global awareness, and financial literacy. Teaching the 
21 st Century Skills requires a variety of teaching strategies such as problem-based learning and 
cooperative learning. States are encouraged to launch initiatives to adopt and implement the 21 st 
Century Skills in their K-12 school system and several states have already begun such initiatives. 
CTE teachers need an understanding of these skills, how they fit within the career and technical 
curriculum, and the contribution CTE can make to ensure that all students possess these skills. 

All Aspects of an Industry refers to (a) the planning, management, financial, technical, and 
production skills and (b) the underlying principles of technology, labor issues, community, and 
health, safety, and environmental issues that students need to enter and succeed in a career field. 
An integral part of Perkins IV, All Aspects of an Industry explores the context in which technical 
skills are used. In addition to the specific skills needed for a particular occupation, the teacher 
who integrates All Aspects of an Industry shows students the big picture of the industry and 
helps students learn the technology, communication, health, and safety issues important to the 
industry as a whole. Beginning teachers who use the States’ Career Cluster Initiative resources 
can find All Aspects of an Industry components integrated in the foundation knowledge and 
skills of the cluster model. 4 

Teachers Need to Know Their Students 

Knowing the career field is not enough to be effective as a CTE teacher; teachers must know the 
students they are charged to teach. Beginning teachers need to “understand how students differ in 
their approaches to learning and create instructional opportunities that are adapted to diverse 
learners” (INTASC, 1992, p. 18). As the NBPTS (1997) noted, “accomplished career and 



4 See http:// www.careerclusters.org . 



99 




technical educators personalize their instruction and apply knowledge of human development to 
best understand and meet their students’ needs” (p. 9). As Lynch (2000) described it, this aspect 
of the knowledge base in education has “exploded enormously” (p. 73). Much more is known 
today about how young people learn, remember, perceive, transfer knowledge, and problem- 
solve. 

Students have certain characteristics based on their developmental level — intellectual, social, and 
emotional — and specific learning needs that vary widely. Knowing students’ stage of 
developmental understanding and the ways in which learning needs vary can help beginning 
teachers plan lessons more effectively. Adolescence is a time of rapid physical maturation, 
intellectual growth, and the development of skills to meet new social demands. Some of the key 
developmental changes occurring during adolescence include: 

• The increasing ability to think abstractly — consider the hypothetical, look at multiple 
dimensions of the same situation, and reflect on themselves; 

• More understanding of internal psychological characteristics — the development of 
friendships based more on perceived compatibility of personal characteristics; 

. Distancing of relationships with parents and family; and 

• Importance of social acceptance, peak of peer conformity. (Steinberg, 2007) 

The students schools serve are becoming more diverse and will likely become more so as our 
nation becomes increasingly diverse (Tomlinson et al., 2004). CTE classrooms in particular have 
a rich history of including diverse groups of students. When planning, teachers should set high 
expectations and provide opportunities for all students to succeed, including boys and girls, 
students with special educational needs, students from all social and cultural backgrounds, 
students of different ethnic groups, and students from diverse language backgrounds. Teachers 
need to be aware that students bring to school different experiences, interests, and strengths that 
will influence the ways in which they learn. A first important step is in taking the time to get to 
know students through questionnaires, personal interviews, and observation. Teachers can also 
learn about their students by reviewing school records, test scores, career interest and skill 
inventories, graduation or career plans, and documents that outline modifications for students 
with special learning needs. 

With careful observation, beginning teachers will notice that students prefer some methods of 
learning over others. These preferences, often referred to as learning styles, can help teachers 
plan instruction more effectively. Students may, for example, prefer to learn by speaking and 
listening; others by what they watch or see; some by touching; and others need to involve their 
whole body in learning. Dispositions for learning can also range along the dimensions of 
approaches to new information. Some students prefer learning that is spontaneous and intuitive; 
whereas others like to plan and organize their work, focusing on a logical progression and details 
that fit in to a larger schematic. A number of frameworks exist for categorizing these learning 
preferences (Felder & Silverman, 1988; Kolb, 1984) and inventories have been developed to 
identify students’ learning styles. CTE teachers can use these assessments to accurately 
determine students’ learning styles and use that information to plan lessons that more closely 
meet the needs of a students. 
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CTE teachers are also responsible for modifying their lessons to accommodate students with 
special learning needs. The Individuals with Disabilities Education Act, designed to ensure that 
students with disabilities have the opportunity to receive a free appropriate public education, 
requires an Individualized Education Program (IEP) for each student who has special learning 
needs. The IEP is a written statement that describes the educational program that is specifically 
designed to meet a child’s individual learning needs. An IEP contains information on the child’s 
achievement and performance, annual goals, support services, how the child is to participate in 
state and district assessments, and what instructional modifications will be provided. CTE 
teachers should consult with special education teachers to learn about how to best support the 
student in learning, including how to use modifications or accommodations such as providing 
more time, shortening the length of assignments, breaking up a large assignment into small parts, 
providing a different format for demonstrating understanding (i.e., verbal instead of written), and 
using tiered assignments that allow learners to work on the same knowledge or skills but with 
varying degrees of complexity, open-endedness, or abstractness. 

CTE teachers should plan their approaches to teaching and learning so that all students can take 
part in lessons fully and effectively. Several key elements of planning for diverse learners 
include: 

• Creating learning environments that value the contribution of all students and where 
students feel secure. 

• Securing students ’ motivation and concentration by using teaching approaches 
appropriate to different learning styles, building on the interests and cultural experiences 
of the students, and planning and monitoring the pace of work so that all students have a 
chance to learn effectively. 

• Providing equal opportunity for all students to participate and including materials that 
allow for a variety of interpretations and outcomes. 

• Using appropriate assessment approaches that provide clear feedback and allow for 
different learning styles and ensure that students are given the chance and encouragement 
to demonstrate their competence and attainment through appropriate means. 

• Setting learning gocds that are attainable yet challenging, to support students’ confidence 
in their ability to learn and to build on students’ knowledge, experiences, interests and 
strengths to improve areas of weakness and demonstrate progression over time. 
(Tomlinson, 2001) 

Teachers Need to Know and Use an Instructional Planning Process 

Good instruction is purposeful and directed toward learning goals. Consequently, the process of 
instructional planning usually begins with identifying what students are to learn. There is a well- 
documented research literature on the importance of clear learning goals (Brophy & Good, 1986; 
Jones, 1992; Walker, 1985) and their appropriateness for the students (Druian & Butler, 1987; 
Kauchak & Peterson, 1987; McCutcheon, 1980; Peterson, Marx, & Clark, 1978; Natriello, 

1987). CTE teachers are required to determine specific instructional goals from a variety of 
sources related to the career field — career cluster frameworks, industry standards, and academic 
standards. In addition, the teacher should consider district curriculum, state CTE standards, and 
industry standards. These instructional goals may deal with knowledge, skills, academic skills, 
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and workplace or employability skills. Good instructional goals must be stated clearly and be 
capable of assessment. The CTE teacher selects and sequences the goals according to the diverse 
students in the CTE program. 

Once CTE teachers have established instructional goals, they are responsible for developing 
instructional plans. The importance of coherent instructional plans is well documented in the 
research literature. Students learn better when the instruction is logically sequenced (Armento, 
1977; Smith & Sanders, 1981). The beginning teacher creates “lessons and activities that operate 
at multiple levels to meet the developmental and individual needs of diverse learners and help 
each progress” and “short-range and long-term plans that are linked to student needs and 
performance” (INTASC, 1992, p. 28). An accomplished CTE teacher “fosters experiential, 
conceptual, and performance-based student learning of career and technical subject matter and 
creates important, engaging activities for students that draw upon an extensive repertoire of 
methods, strategies, and resources. Effective practice is also marked by the ability to integrate 
career and technical and academic disciplines productively” (NBPTS, 1997, p. 39). 

Instructional planning usually involves “chunking” like goals into units and then into daily 
lessons. An instructional unit includes a logical sequence of learning activities that will lead to 
students achieving the instructional goal and an assessment that provides evidence of the 
learning that is achieved. Each learning activity has a purpose in the unit as a whole and has a 
reasonable time frame. The learning activities in the unit should be selected so as to be suitable 
for the students’ learning needs. Unit design also involves selecting appropriate materials and 
resources to support the learning goal. 

The process of unit planning has been heavily influenced by the standards movement, which 
focused educators on helping all students achieve a well-defined set of outcomes from their 
learning. The concept of backwards design is based on beginning with the end, or in this case, 
the standards, in mind. Backwards design suggests a planning sequence in which the first step 
following the identification of learning goals is to develop the assessment that will be used to 
determine whether those learning goals have been achieved (Wiggins & McTighe, 1998). Once 
the acceptable evidence has been determined for the assessment, learning experiences and 
instruction are planned. This design concept has been detailed in a wide variety of formats, 
including SREB’s Planning for Improved Student Achievement: Ten Steps for Planning and 
Writing Standards-Based Units (2007). 

In addition to planning units of instruction, CTE teachers at the secondary level are also required 
to plan a course syllabus. Planning at the course level can increase the quality of CTE programs 
(Bottoms, Pucel, & Phillips, 1997). In SREB’s Designing Challenging Vocational Courses: A 
Guide to Preparing a Syllabus (Bottoms et al., 1997), instructors are encouraged to: decide what 
knowledge, understandings and skills are needed for the career field; set high standards; use 
projects as a focus for learning; establish their role as a facilitator of learning; and assess 
continuously. The steps in the syllabus design process are to: 

• Describe the course 

• Clarify the instructional philosophy 

• Determine major course goals 

• Select and put into sequence major course projects 
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• Develop project outlines 

• Decide on an instructional delivery plan 

• Develop an assessment plan for the course (pp. 35-61) 

In anticipation of the need to communicate CTE standards-based instructional plans to other 
teachers as part of the effort to integrate academic and CTE content, CTE teachers also use a 
process called curriculum mapping. Curriculum maps are based on the school calendar and 
outline essential knowledge and skills to be examined, processes and skills to be emphasized, and 
products and performances that are the assessment of the learning (Jacobs, 1997). Curriculum 
maps allow teachers to record the time it takes to teach particular content and to analyze those 
time frames in reference to other units of study and the whole curriculum of a course. The unique 
contribution of curriculum maps as a standards-based instructional tool for CTE teachers is that 
they provide a visual sequence of what is taught and when it is taught so that the information can 
be shared with other teachers. Such information is central to the process of identifying points in 
the curriculum where integration opportunities exist. 

There is a well-documented link between teacher effectiveness, planning of learning activities, 
and selection of materials for learning (Clark & Yinger, 1979; Emmer, Sanford, Clements, & 
Martin, 1982; Everston, Anderson, Anderson, & Brophy, 1980; McCutcheon, 1980; Peterson et 
al., 1978). Beginning teachers can evaluate teaching resources and curriculum materials for their 
comprehensiveness, accuracy, and usefulness for representing particular ideas and concepts 
(INTASC, 1992). CTE teachers need to be aware of and understand how to access a variety of 
resources for instructional planning including school, district, or industry resources, experts from 
the community or career field, and commercial products. Effective CTE teachers select, adapt, 
and create resources that meet criteria for quality. Using multiple resources provides many 
avenues to student understanding. 

Teachers Need to Be Reflective About Their Practice 

Studies have documented the value of teacher reflection, both alone and in collaboration with 
other teachers (Borko, Lalik, & Tomchin, 1987; Borko & Livington, 1989; Borko, Livington, 
McCaleb, & Mauro, 1988; Colton & Sparks-Langer, 1992, 1993; Ellwein, Graue, & Comfort, 
1990; Ross & Regan, 1993; Tabachnick & Zeichner, 1991). “The beginning teacher is a 
reflective practitioner who continually evaluates the effects of his/her choices and actions on 
others (students, parents, and other professionals in the learning community) and who actively 
seeks out opportunities to grow professionally” (INTASC, 1992, p. 31). 

According to the NBPTS (1997), “accomplished career and technical educators regularly 
analyze, evaluate, and strengthen the effectiveness and quality of their practice through lifelong 
learning” (p. 61). Thinking back on a teaching experience helps teachers understand what 
happened (effective and ineffective learning episodes) and how that experience can affect their 
future practice. Beginning teachers especially need to cultivate the skill of reflecting on teaching 
and the criteria that makes a lesson or unit successful. 

Reflective practice includes several components. In one sense, the teacher purposefully uses 
classroom observations, classroom assessment data, and other information from the students to 
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revise and improve instruction. Reflection considers whether or not the students achieved the 
learning goal and why. Reflective practices can also include seeking out professional literature 
and resources for professional growth as well as drawing on colleagues for feedback, problem- 
solving, and new ideas. Basic questions to guide reflection include: 

• What am I doing in the classroom? 

• Why am I doing this? 

• What is the result? 

• Will I change anything based on the information gathered from answering the first 
three questions? 

The NBPTS Career and Technical Standards (1997) suggest that reflection includes three 
important components: evaluating results and seeking input from a variety of sources; reflecting 
on one’s own point of view; and continually refining practice through study and self- 
examination. As the fields in which CTE teachers work change over time, they need to stay 
abreast of new information and trends and demonstrate that they are continuing to pursue up-to- 
date knowledge. 

Teachers Need to Collaborate with Colleagues and Other Stakeholders 

There is clear evidence that higher student achievement is linked to collaboration (Goddard, 
Goddard, & Tschannen-Moran, 2007). The beginning teacher “fosters relationships with school 
colleagues, parents, and agencies in the larger community to support students' learning and well- 
being” and “participates in collegial activities designed to make the entire school a productive 
learning environment” (INTASC, 1992, p. 33). Working in collaboration with others is perhaps 
even more important for CTE teachers. The NBPTS states that “accomplished career and 
technical educators work with colleagues, the community, business and industry, and 
postsecondary institutions to extend and enrich the learning opportunities available to students 
and to ease school-to-work transitions” (p. 65). 

CTE teachers partner with business, industry, labor, and the community in a variety of ways. 
Beginning teachers usually foster these partnerships through a program advisory committee. The 
committee is composed of members from business, industry, and the community who can 
provide assistance in creating a CTE program that meets the needs of the students and business 
and industry in the community. The teacher uses recommendations made by the committee to 
design, develop, operate, assess, and support the CTE program. These partnerships ensure that 
the CTE classroom is well aligned with the realities and demands of the workplace. Partners 
inform curriculum, the CTE facilities, and work-based learning opportunities. 

CTE teachers collaborate with other educators to integrate academic and career and technical 
education. This partnership is critically important to the new mission of career and technical 
education. Bragg and Mills (2005) suggested that, given the current policy context and emphasis 
on accountability, integration efforts will predominantly occur with core academic curriculum 
areas — reading and writing (English), mathematics, and science, and to a lesser degree, social 
studies. Beginning CTE teachers need to be prepared to deal with a variety of integrated learning 
approaches (Bottoms & Sharpe, 1996). At the very least, beginning teachers will use single 
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course integration and embed academic concepts, ideas, or processes into their CTE instruction. 
This may require minimal collaboration. More collaborative integration approaches include joint 
planning across or within departments, team teaching, thematic projects or units, and 
organizational approaches as career academies. To get started in collaborating with academic 
teachers, CTE educators can: 

• Review learning goals for CTE and academic programs and look for connections; 

• Develop and implement an integrated learning plan, including a description of the 
projects, instructional strategies and assessments they will use to measure student 
learning; and 

• Review, revise, and improve integrated learning activities (Bottoms & Sharpe, 
1996, pp. 98-106). 

CTE teachers collaborate with postsecondary partners to articulate courses and provide a 
seamless transition from high school CTE studies to postsecondary education and training 
programs. These partnerships are important because they enable students to leave high school 
with a clear focus and ready to learn at the college level without remediation (Bragg & Mills, 
2005). It is important for today’s students to have multiple entry and exit points into and out of 
the secondary and postsecondary educational system. Consequently, beginning teachers must 
have a grasp of the importance of dual credit or articulation agreements, industry certifications 
for CTE programs, and the relationship between two-year and four- year college and university 
programs in the career pathway in which they teach. 
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Appendix 2B 



Concept Paper for Instructional Strategies Module 

Beginning teachers need to understand and know how to use a variety of instructional strategies 
to encourage students' development of critical thinking, problem solving, and performance skills 
(INTASC, 1992). Facing the challenge of preparing students for both college and careers, expert 
CTE teachers need effective strategies to actively engage students, boosting both motivation and 
achievement. Historically, many CTE programs were based on detailed, competency-based 
curriculum in which students practiced and repeated skills (Lynch, 2000). Such an approach is 
insufficient in preparing students for the 21 st century workplace. Cognitive science asserts a 
theory of learning that points to a wider variety of instructional methods aimed at helping 
students learn new information based on their prior knowledge and effective instruction that 
helps students uncover existing knowledge, including misconceptions. This approach, referred to 
as constructivist teaching, results in classrooms where learning is: 

• Constructed: Students come to learning situations with already formulated knowledge, 
ideas, and understandings. Students integrate new experiences and interpretations to 
construct their own personal meaning with this previous knowledge. 

• Active: The student is the person who creates new understanding for her/himself. The 
teacher guides knowledge, but allows the students to experiment, manipulate objects, ask 
questions and try things that don't work. Students also help set their own goals and means 
of assessment. 

• Reflective: Teachers should create opportunities for students to question and reflect on 
their own learning processes, either privately or in group discussions. The teacher creates 
activities that lead the student to reflect on his or her prior knowledge and experiences. 

• Collaborative: The constructivist classroom relies heavily on collaboration among 
students because students learn about learning not only from themselves, but also from 
their peers. When students together review and reflect on their learning processes, they 
can pick up strategies and methods from one another. 

• Inquiry- or Problem-Based: The main activity in a constructivist classroom is solving 
problems. Students use inquiry methods to ask questions, investigate a topic, and use a 
variety of resources to find solutions and answers. 

Evolving: Students have knowledge that they may later see as incorrect or insufficient to 
explain new experiences. As students explore a topic or problem, they draw conclusions, 
and, as exploration continues, they revisit those conclusions and modify them to support 
new knowledge or experiences. (Singer & Moscovici, 2008) 

Consistent with the vision of CTE instruction outlined in the introduction to this paper, 
constructivist teaching requires the use of a few instructional strategies on which CTE teachers 
can focus. Each of the following parts of this section will describe one of these strategies, the 
rationale for including it among the most essential strategies beginning teachers need to know 
and be able to do, and key information that teachers need to know to implement that strategy 
effectively. 
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Project- and Problem-Based Learning 

Project-based learning (PBL) is an instructional model that organizes learning around projects — 
complex tasks based on challenging problems or questions. Through PBL, students are engaged 
in design, problem-solving, decision-making, or investigative activities that give them the 
opportunity to direct their own learning and that result in a product or presentation (Thomas et 
al., 1999). Although projects may vary in duration and scope, recent definitions of PBL 
recognize that good projects are standards-focused, engaging students in “learning knowledge 
and skills through an extended inquiry process structured around complex, authentic questions 
and carefully designed products and tasks” (Buck Institute for Education, 2003, p. 4). 

In CTE, PBL is used to bring an authentic, motivating context to teaching career-related 
competencies. A CTE project may involve workplace simulations such as designing, assembling, 
testing, or evaluating a product; developing and/or implementing a plan to meet a local business 
need; or operating a business in the school or community (Bottoms et al., 1997). Projects usually 
require a series of steps and provide opportunities to make judgments and decisions when 
unexpected events occur, typical conditions found on the job. Most importantly, projects teach 
students to resolve workplace problems, fostering the development of the essential problem- 
solving skills needed for the workplace. 

The varying nature and depth of PBL implementation complicates the research base on its 
effectiveness as an instructional method. Projects vary widely in the depth of the content 
covered, duration, and the degree to which they are student-directed. Teachers cite a variety of 
reasons for not using projects, including lack of time, resources, and technology. In spite of these 
limitations, Thomas’ (2000) comprehensive review of research on PBL found some evidence 
that the approach enhances the quality of student learning when compared with other 
instructional methods. PBL has the potential to help students learn not only subject matter 
content, but the ability to put that content to use in real-life situations to solve problems and 
make decisions. This goal is particularly important in CTE classrooms because students are 
solving problems and completing projects similar to those they will face as they enter and 
advance in the workplace. 

PBL teaches students complex processes and procedures such as planning and communicating 
and leads to higher-level cognitive development through active engagement with complex 
situations (Buck Institute for Education, 2003). Teachers report that PBL: 

• Helps bridge the gap between “knowing and doing,” or knowledge and thinking. 

• Encourages the development of habits of mind associated with learning throughout life, 
civic responsibility, and career success. 

• Assesses performance on content and skills using criteria similar to those in the work 
world, thus encouraging accountability, goal setting, and improved performance. 

• Creates positive communication and collaborative relationships among diverse groups of 
students. 

• Meets the needs of learners with varying skill levels and learning styles. 

• Engages and motivates students who are typically unmotivated. (Buck Institute for 
Education, 2003, p. 6) 
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In order to successfully implement PBL and reap its benefits as an instructional strategy, Thomas 
(2000) offered a set of characteristics that should be present: 

• Projects must be central, not peripheral to the curriculum. In a PBL instructional 
approach, projects represent the central learning concepts, knowledge and skills of the 
subject. In CTE, that means selecting projects that enable students to learn career 
competencies. Problem-based learning specifically draws problems and scenarios 
exclusively related to the career field (Bottoms et al., 1997). 

• Projects are focused on questions or problems that lead students to learn the centred 
concepts and principles of a discipline. Teachers are challenged to define projects so that 
the learning activities connect to the concepts and skills that the students are expected to 
learn. This can be accomplished with an essential question or, in the case of problem- 
based learning, an ill-defined problem. Thomas (2000) warned that PBL projects may be 
built around thematic units or the intersection of topics from two or more disciplines, but 
that is not sufficient to define a project. The questions that students pursue, as well as the 
activities, products, and performances that occupy their time, must be designed to achieve 
an important purpose. 

• Projects involve students in a constructive investigation. An investigation is goal- 
directed, involving inquiry, knowledge-building, and resolution. In order to be considered 
as a PBL project, the activities that are central to the project must lead students to 
construct their own knowledge (new understanding, new skills). If the project activities 
are of no difficulty to the student or the project can be accomplished without the 
application of already-leamed information or skills, the project is an exercise, not a PBL 
project. 

• Projects are student -driven to some significant degree. PBL projects are not highly 
prescribed nor structured to be led by the teacher. Laboratory exercises and learning 
packets are not examples of PBL, even if they are problem-focused and central to the 
curriculum. PBL projects allow for more student autonomy, choice, and student 
accountability than traditional instruction and traditional projects. 

• Projects are realistic, not school-like. Projects embody characteristics that give them a 
feeling of authenticity. In CTE, the projects are drawn from major activities exclusively 
related to the career field. They should represent the culminating activities of large units 
of study rather than narrow sub-skills or tasks, (pp. 3-4) 

Problem-based learning is closely related to project-based learning in that both involve complex 
tasks that engage students in planning, gathering and evaluating information, analyzing situations 
and developing solutions. The difference between the two lies in the specific aspects of delivery. 
Problem-based learning begins with an ill-defined problem and asks the student to hypothesize 
how to solve it (Delisle, 1997), which cannot be said of all projects. Once the problem is defined, 
students access, analyze, and use data and information from different sources, revise the initial 
hypotheses as needed, and develop and justify solutions according to evidence and reasoning 
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(Barrows, 1986; Gallagher, Stepian, Sher, & Workman, 1995). The problem-based learning 
approach, used in medical schools to develop clinical reasoning, is outlined by Barrows and 
Tamblyn (1980) as follows: 

1. The problem is encountered first in the learning sequence, before any preparation or 
study has occurred. 

2. The problem situation is presented to the student in the same way it would present in 
reality. 

3. The student works with the problem in a manner that permits his or her ability to reason 
and apply knowledge to be challenged and evaluated, appropriate to his or her level of 
learning. 

4. Needed areas of learning are identified in the process of work with the problem and used 
as a guide to individualized study. 

5. The skills and knowledge acquired by this study are applied back to the problem, to 
evaluate the effectiveness of learning and to reinforce learning. 

6. The learning that has occurred in work with the problem and in individualized study is 
summarized and integrated into the student's existing knowledge and skills, (pp. 191— 
192) 

Because it is driven by authentic situations, problem-based learning is both rigorous and 
relevant, engaging students in a highly participatory learning experience. Students become 
interested in the problem they are solving and are more motivated to learn. Through their active 
engagement in the learning process, students take responsibility for their own learning and are 
better able to define topics, access resources, and evaluate the validity of those resources 
(Gallagher et al., 1995; Krynock & Robb, 1996). Sungar and Tekkaya (2006) conducted a quasi- 
experimental study of two biology classrooms, one taught with teacher-centered, textbook- 
oriented traditional instruction and the other with problem-based learning. The students who 
received problem-based learning had higher levels of critical thinking, metacognition, and effort 
than the students taught with traditional methods. The biggest impact, however, was observed on 
the students’ ability to self-regulate their own learning, actively sustaining their thoughts, 
behaviors, and emotions to reach their goals. Other studies have shown problem-based learning 
to improve critical thinking, communication, mutual respect, team work, and interpersonal skills 
(Achilles & Hoover, 1996; Gordon, Rogers, Comfort, Gavula, & Mcgee, 2001; McBroom & 
McBroom, 2001; Sage, 1996; Savoie & Hughes, 2004; West, 1992). Problem-based learning has 
significant potential to teach the kind of skills that CTE students will need to succeed in further 
learning and the workplace. 

John Mergendoller of the Buck Institute for Education asserts (personal conversation, February 
2009) that designing project- or problem-based learning presents a real design challenge to 
teachers, and certainly to CTE teachers entering the field under the challenges and limitations of 
an alternative certification program. The Buck Institute is currently using a project planning form 
that can be helpful. The goal of the form is to help teachers see project-based learning as an 
“envelope” into which all other strategies fit — to help students really leam how problems and 
questions can drive learning and help them take well-thought-out, effective action in response to 
real-life situations. Given the nature of CTE content, it is likely that most, if not all, units of 
study may be planned with a driving project or problem. If real workplace problems, questions, 
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or scenarios drive student learning, students are much more likely to reap the benefits of this 
strategy — problem-solving and higher order thinking skills, as well as the ability to regulate their 
own learning throughout life. 

Cooperative Learning 

The purpose of cooperative learning (CL), similar to the purpose of project- or problem-based 
learning, is to make each student a strong, independent learner. Students who experience 
teaching that allows them more control learn more and enjoy learning more (Ramsden, 2003; as 
cited in Hutchinson, 2007). According to Abrami et al. (1995; see also Abrami, Chambers, 
Poulsen, & Chambers, 2004) and Laverie (2006), evidence suggests that CL promotes student 
achievement and the development of social and interpersonal skills. Johnson, Johnson, and Smith 
(1991; as cited in Flowers & Ritz, 1994) synthesized over 375 studies on the effect of 
cooperative, competitive and individualistic efforts on student achievement and productivity. 
They found that students in cooperative learning settings performed better than students in either 
competitive or individualistic settings. They also noted that cooperative learning "resulted in 
more high-level reasoning, more frequent generation of new ideas and solutions (i.e., process 
gain), and greater transfer of what is learned within one situation to another (i.e., group to 
individual transfer) than did competitive or individualistic learning" (p. 212). 

CL is a particularly important strategy for CTE teachers. Sutphin and Newsom-Stewart (1995) 
found that students enrolled in agricultural education identified activity-centered learning, 
opportunities for work experience, and the development of teamwork and life skills as reasons 
for enrolling in that CTE course. Students gravitate to active learning that engages them socially 
as well as intellectually, as illustrated in the popularity of Facebook and other forms of social 
networking among today’s youth. CL may be particularly effective when integrated with project- 
or problem-based learning. Roger Johnson of the Cooperative Learning Institute asserts that CL 
is a process for doing things in classrooms — raising achievement, building self-esteem, 
encouraging positive attitudes, and building acceptance of differences in diverse classrooms — 
although project-based learning is a good content home for building cooperative team 
relationships (personal conversation, February 2009). CL can also influence classroom 
management in a positive way. The classroom environment is much more productive and 
positive when students are concerned about each others’ behavior and learning. Peers may be 
one of the most important influences on each others’ success. 

As a teaching practice, CL can present unique challenges for both the practitioner and student. In 
order to implement CL effectively, teachers must plan carefully (Hutchinson, 2007).Despite the 
evidence that CL is an effective teaching practice, the use of CL varies widely among teachers — 
from 10% to 93% (Antil, Jenkins, Wayne, & Valdasz, 1998, as cited in Abrami et al., 2004). To 
add to the complexity, there are at least four types of CL: informal (with direct teaching), formal 
(to work through projects and assignments), base groups (long term academic and personal 
peer support groups) and cooperative controversy groups (to structure intellectual 
argument). One of the first challenges in effectively using CL is considering the goals of 
instruction and figuring out what type of CL to use. All four types are useful in 
different ways. For example the informal CL strategy improves direct teaching by interspersing 
short, active student conversations/review of the material being delivered during the direct 
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instruction. The following chart reflects the essential components of CL that teachers must use to 
implement any type effectively. 

Essential Components of Cooperative Learning 

1. Positive Interdependence: “We all sink or swim together” 

• Each team member’s efforts are required for team success. 

• Each team member has a specific and unique contribution because of his/her resources, talents, and 
task responsibilities. 

2. Face-to-face interaction: “Students become translators” 

• In cooperative learning teams, students promote each other’s success by sharing resources and helping, 
supporting, encouraging, and celebrating each other’s efforts. 

• Teachers structure teamwork so that students help each other by explaining how to solve problems, 
teaching one’s knowledge to others, checking for understanding, discussing concepts, and connecting 
present and past learning. 

3. Individual accountability / personal responsibility 

• Each team must be accountable for achieving its goals, and each member must be accountable for 
contributing his or her share of the work. 

• Each student is individually assessed and the results are given back to the team and individual to 
determine who needs more assistance and support for learning. 

4. Interpersonal and teamwork skills: “Social skills do not magically appear” 

• Social skills must be taught just as academic skills are taught. 

• Leadership, decision-making, trust-building, communication, and conflict management are essential to 
team success. 

5. Team reflection: “How are we doing as a team?” 

• Teachers need to structure teamwork so that team members discuss how well they are achieving their 
goals and how effectively they are working together. 

• Teams should describe what member actions are helpful and unhelpful and then make decisions about 
what behaviors to continue or change. This is an ongoing process of self assessment and peer 
assessment. 



Teamwork / Social / Academic Skills 

Must be modeled, taught, rehearsed, practiced, used, reinforced, and the skill must mean something. Teach 
students what it “looks like,” what it “sounds like” and what it “feels like.” Students will learn more and do better 
if they help their classmates, if they ask for help when they need it, and if they get an explanation with the help. 

Teach students how to help each other - notice when a team member needs help; tell your team members to ask 
you if they need help; when someone asks for help, help them; don’t give answers - give explanations; praise and 
encourage; and, check to make sure they understand. 



Signs of Success 



Students are experiencing success if: 

□ students drill each other on the material. 

□ answers are shared (explanations). 

□ materials are shared. 

□ heads are close together. 

□ students give their opinions easily and candidly. 

□ social skills improve - in teams and elsewhere. 



Because the teacher plays an essential role in CL — structuring the classroom environment to 
facilitate learning in a group (Smith & McGregor, 1992) — there is considerable instructional 
planning that needs to occur to create CL lessons. Cooperative learning provides the “directions 
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for learning, where the focus is on the product of learning (Myers, 1991, as cited in Hutchinson, 
2007, p. 358) — the accomplishment of an end goal or learning outcome that is closely controlled 
by the teacher” (Cooper & Robinson, 1998, as cited in Hutchinson, 2007; see also Rockwood, 
1995). In other words, the teacher structures the environment, monitors the progress, and 
provides a theoretical foundation to enable students to reach the learning goals. The teacher 
moves from being the sole expert in the class to an external consultant in the learning 
environment with students being responsible for the achievement and implementation of their 
group objectives. This does not happen by accident, but is the result of teaching the CL concepts 
to students, practicing the habits of CL with effective feedback, and structuring the learning 
environment with appropriate activities and resources. 

There is also the potential for teachers to play a more frequent and active assessment role in the 
CL classroom. When students work actively in groups, the teacher is free to observe and monitor 
groups, gathering data about what is going on and providing timely feedback to the class, the 
groups, or to individual students. This role provides the teacher with a “window into the 
students’ minds” through close observation of learning that is taking place out loud in group 
conversation and interaction (Roger Johnson, personal conversation, February 2009). 

The teacher is also responsible for facilitating frequent reflection on group functioning. To 
maximize the benefits of CL, groups need to regularly reflect on how well they are working 
together. This feedback loop is known as group processing. It is critical to enhancing group 
performance and is analogous to evaluation and team-building efforts that occur in industrial 
settings. Processing can be facilitated by an evaluation form on which each team member 
evaluates others on the criteria outlined in their group covenants and completes certain key 
statements (Towns, 1998). 

Cooperative learning is a technique that has proven effective in many settings to improve student 
learning and social skills. It must be practiced in order to be proficient. Slavin (1991) noted that 
the widespread and growing use of CL: 

demonstrates that CL methods are practical and attractive to teachers. The 
history of the development, evaluation, and dissemination of cooperative 
learning is an outstanding example of the use of educational research to 
create programs that leave improved the educational experience of 
thousands of students and will continue to affect thousands more. (pp. 81) 

Effective Strategies for Presenting Information 

As CTE teachers use project- and problem-based learning integrated with the principles of 
cooperative learning, there will be instances when students need new information to further their 
learning. Although students will learn how to access sources of information on their own as part 
of project- and problem-based learning, sometimes the most efficient strategy for the CTE 
teacher is to present information to all students at once. When these opportunities occur, CTE 
teachers need strategies to present that information in ways that are effective, meaningful, and 
engaging to students. Beginning CTE teachers may have limited understanding of ways to 
present information, particularly if their own experience largely included typical methods such as 
lecturing, reading aloud together, and completing study questions at the end of the textbook 
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chapters. Consequently, they need to be aware of effective strategies for presenting information 
based on research and the collective wisdom of practitioners, such as: 

• Following a series of steps in planning a lesson focused on the presentation of information, 
including: 

(a) establishing instructional set for a few minutes by simply introducing what it is 
students are to learn (objective), 

(b) describing why it is important to learn the information (relevance), 

(c) outlining how students will learn the information (what the teacher and the students 
will do), and 

(d) describing how the information will be assessed (project, problem to solve, paper, 
application, “homework,” etc.). 

• Using technology and presentation software and supplements to support nearly every 
presentation (i.e., direct instruction) where new information (i.e., direct instruction) must be 
provided. There are many technology applications available, including: PowerPoint, Excel, 
LCD panel, scanner, graphics and information downloaded from the Internet, digital 
cameras, videos, e-mail, web-based materials, and interactive white boards. Creating tutorials 
for students who need extra help, additional content, individualized study, or assignments can 
also be fairly easy using available software and Internet resources. 

• Drawing on metacognitive processes such as reading technical materials with students, 
asking open-ended questions where they have to create or explore responses, working with 
them on technology applications, providing guided and independent practice, and having 
them summarize content they’ve read and put forth its implications for their employment or 
projects. 

• Presenting relevance for nearly all points, such as examples from the community, clips from 
the newspaper, an analogy, rules or guidelines, case studies from the literature, illustrative 
material from textbooks, a math problem or science demonstration used in industry or 
medical fields, a scenario, a prediction (“a” should lead to “b”), or ask them to provide an 
illustration from their own experience. 

• Acquiring a repertoire of classroom methods that facilitate learning through social 
interaction, often referred to as scaffolding. Illustrative examples include question and 
answer dialog, cooperative learning strategies (described earlier in this paper), small-group 
discussions, buzz groups, brainstorming, classroom workshop, teacher or peer demonstration 
of a skill or habit, student repetition of that skill or habit, games, debates, and tutoring. 

• Checking frequently on learning. This can often be done through questions and answers, a 
“quick check” quiz, providing guided and independent practice, conducting and writing up an 
experiment or similar assignment, repeating or practicing a demonstration, solving work- 
related math problems, or any other context-related application of the content that was 
presented. 

Integration of Academic Content and Skills 

Perkins IV requires schools to develop career and technical education programs with challenging 
academic and technical standards and requires CTE instructors to provide challenging instruction 
that integrates academics with technical knowledge and skills. This policy context reflects the 
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needs of modern workplaces. A survey of human resources professionals indicates that “far too 
many young people are inadequately prepared to be successful in the workplace. At the high 
school level, well over one-half of new entrants are deficiently prepared in the most important 
skills — oral and written communications, professionalism/work ethic and critical 
thinking/problem solving” (Conference Board, 2006, p. 7). This survey alone highlights the 
importance of preparing CTE students to be academically successful while ensuring students 
possess technical knowledge and skills. 

The current literature illustrates the importance of integrating CTE and academic studies in 
student preparation for high school graduation and readiness for postsecondary education and 
careers. “At the high school level, career and technical education programs provide the most 
effective learning opportunities. Not only are students applying skills and knowledge to real- 
world situations in their career and technical programs, but also they are drawing on knowledge 
learned in core subjects” (Daggett, 2005, p. 5). The SREB’s High Schools That Work and 
Technology Centers That Work survey of 12 th -grade students reveals that CTE students 
experiencing an intensive emphasis on integrating academic content and skills into CTE courses 
meet college- and career-readiness goals at a much higher rate than those who experience low or 
moderate emphasis on academic integration (SREB, 2008). A prominent recent research study 
conducted by the NRCCTE (Stone et al., 2006) placed a CTE teacher with a mathematics teacher 
to develop lesson plans with integrated mathematics. This study provided evidence that an 
integrated curriculum leads to higher test scores. CTE students taught a curriculum with 
integrated mathematics outscored the control group on two tests of mathematics ability. 

The challenge in academic and CTE integration is to meet the needs of its CTE teachers who are 
eight times more likely than their academic counterparts to lack academic training in the form of 
a bachelor’s degree or specific subject knowledge (NCES, 2000). In addition to this lack of 
academic preparation, there are other difficulties that plague integration efforts, particularly for 
beginning CTE teachers (Hoachlander, 1999). First, integration can result in a loss of focus if the 
objective is not clear. Second, integrating academic and career and technical curriculum depends 
on more than simply identifying work-related applications of academic knowledge and skill. It 
also involves ensuring the use of a variety of instructional strategies that are engaging to all 
students. Finally, constructing a rich, complex, cumulative integrated curriculum that 
simultaneously helps students master an academic discipline and apply it in a coherently defined 
domain of the work world demands time, expertise, and resources that are beyond the reach of 
most teachers. 

A beginning CTE teacher can use several approaches to the integration of academic content 
(Drake & Bums, 2004). First, the CTE teacher can choose academic content to integrate into his 
or her own course, a transdisciplinary approach. An interdisciplinary approach can also be used 
in which the CTE teacher works with academic teachers to organize learning around common 
objectives across disciplines. A multidisciplinary approach calls for teachers to organize 
standards from each discipline around a theme. Drake and Bums (2004) described these 
approaches along a continuum. As teachers select approaches along the continuum, the 
transdisciplinary approach is least complex and requires the least time for collaboration whereas 
the multidisciplinary approach is most complex and requires the most time for collaboration. 
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Another alternative integration approach for beginning CTE teachers is to use a contextual 
approach, adding the academic content that is used within real-world career contexts. There is 
evidence that this approach, researched in a recent study on academic and CTE integration by the 
NRCCTE (Stone et al., 2006), increases CTE student attainment of academics without lowering 
achievement on CTE competencies. Whether teachers use one of the approaches suggested by 
Drake and Burns (2004) or a contextual approach with the academic content used in real-world 
contexts, there is a need to embed both literacy and numeracy in order to help students prepare 
for further learning and the workplace. 

Embedding Literacy 

Increasingly U.S. jobs — even the shrinking pool of blue-collar jobs — require and depend upon 
reading. A generation ago, jobs in factories, foundries, and mills commonly required no reading, 
and many other jobs (e.g., law enforcement, practical nursing, trucking) required reading in 
limited amounts, but this has changed. The problem occurs when students are not taught to read 
the texts they will encounter in the professional world. Researchers, such as Timothy and 
Cynthia Shanahan (2008), have become increasingly concerned that students are only learning 
reading skills that can be generalized in all reading situations. The research into “disciplinary 
literacy” is just beginning. 

In a two-year study of the skills required for various disciplines, the Shanahans (2008) concluded 
that the high-level skills and abilities embedded in these disciplinary or technical uses of literacy 
are probably not particularly easy to learn nor are they often taught. The Shanahan study 
concluded that, as students move through school, reading and writing instruction should become 
increasingly disciplinary, reinforcing and supporting student performance with the kinds of texts 
and interpretive standards that are needed in the various disciplines or subjects. Although the 
study does not include CTE classes, the principles apply to all content areas. 

Additional research concludes that many people learn better and faster, and retain information 
longer, when they are taught concepts in context. Sticht, Armstrong, Hickey, and Caylor (1987) 
found that teaching young soldiers who lacked even basic literacy skills to read in the context of 
their daily tasks not only increased their competency in those tasks but also improved their 
general reading skills — all in a relatively short time period. 

ACT’s (2006) study of skills necessary for the workplace and for college, as measured by ACT 
and WorkKeys, shows that skills transfer into both settings. The essential workplace skills are as 
follows: 

■ Understand main ideas, topic sentences, and the relationships among sentences in a 
paragraph 

■ Correctly use technical terms when describing the main idea and supporting details in a 
passage 

■ Recognize organizational structures of passages to identify pertinent details and recognize 
appropriate applications 

■ Select important details to clarify meaning 

■ Apply straightforward instructions to new situations 
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■ Apply complex instructions that include conditionals to situations described in a passage 

■ Figure out the correct meaning of a word based on how the word is used 

■ Understand the definitions of acronyms defined in a passage 

■ Identify the appropriate definition of words with multiple meanings based on context 

■ Apply technical terms to situations 

■ Apply given information to new situations (pp. 4-5) 

Although the skills can be defined, the strategies to assist students in gaining these skills are not 
as clear. In CTE classrooms, the problem is compounded by how the teachers view themselves 
as readers and their lack of training. 

Through a series of interviews with 35 CTE teachers from 1997 to 2004, Darvin (2006) found 
that the instructors did not view themselves as readers. She concluded that the teachers used a 
wide variety of texts, but “differently.” They saw themselves as poor readers because they did 
not have long-term comprehension, did not read entire books in a linear fashion, and did not 
read quickly. 

All of the teachers in the study did use texts frequently to consult for information. Students 
moved frequently from text to task and back again in their authentic applications. Darvin 
referred to this practice as “situated literacy.” She concluded that the purposes for reading in 
CTE classes are to gather information, construct meaning and apply knowledge to solve 
particular problems and create new ideas. Within that context, teachers must be adept at helping 
students process different kinds of texts. 

Miller and Miller (2003) found similar situations in technical classes in college. Although 
instructors generally have a substantial depth of knowledge and a high degree of technical skill, 
they are not necessarily effective in the instructional role. The instructor, by nature of the job, 
must be able to assist a beginner. Sometimes the instructor’s high levels of competence may 
interfere with instructional effectiveness rather than contribute to it. 

As the Shanahan study noted, professionals use discipline-specific strategies that are not always 
easy to identify. ACT has identified the most important skills for the workplace. However, there 
is not yet a research base to identify specifically which strategies are most effective to teach 
these essential skills. 

Embedding Numeracy 

Consideration of current research highlights the fact that students today learn differently than 
their pre-technology counterparts, which has implications for the way in which CTE teachers 
should embed numeracy in instruction. Patricia Greenfield referred to these post-technology 
students as the “game generation” and her research identified specific cognitive skills unique to 
them. In addition to an increased ability to multi-task, they are more comfortable with visual- 
spatial skills, mental maps, and seeing the computer as a tool and learn through trial and error, 
observation, and hypothesis testing (Greenfield, 1984; see also Barkley, 2007). This type of 
learner requires alternatives to the strategies commonly seen in the traditional classrooms of the 
baby boomer generation. Prensky argued this need is due to replacement of the sequential or 
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linear thinking, predominantly seen in the baby boomer generation, by “parallel processing” 
(Prensky, 2000, pp. 2-17). 

The traditional model of teaching mathematics emphasizes learning mathematical concepts in 
isolation. Teaching mathematics contextually involves providing students the opportunity to 
learn math concepts embedded within an application scenario. “The contextual approach 
recognizes that learning is a complex and multifaceted process that goes far beyond drill- 
oriented, stimulus-and-response methodologies” (Center for Occupational Research and 
Development [CORD], 1999, p. 1). This method of teaching and learning in context meets the 
need of the “game generation” to learn globally and incorporates multiple forms of experience so 
that students may find meaningful connections between application and abstract understandings. 

A recent study by the NRCCTE found the Math-in-CTE model, which involves enhancing the 
mathematics naturally embedded in CTE curriculum, significantly improved student math 
achievement (Stone et al., 2006). CTE instructors typically understand how to use the 
mathematics of their career pathway, but are not trained on the mathematical theory behind it. 

For example, construction teachers consistently teach students to use the 3, 4, 5 angle concept to 
square up the comers of a foundation or wall frame but do not connect this procedure to the 
Pythagorean Theorem, which is the basis of why it works. In order to enhance embedded CTE 
mathematics and better help students to master these concepts, teachers must understand the 
connections, vocabulary, and theory of the math unique to their CTE area. 

Unfortunately, students often cannot transfer knowledge from one situation to another because 
the CTE concepts and the math concepts are chunked, making it difficult to recognize and 
separate them (Karweit, 1993). True contextual learning must include a deliberate effort to 
connect what is learned experientially to not only other contextual examples but also to the 
theoretical principles behind these authentic activities (Fuchs et al., 2003). Without these 
connections to the abstract mathematics, students are unable to transfer their new knowledge to 
new situations or to their general math understanding (Boaler, 1993, 1998; Lave, 1988; Lave & 
Wenger, 1991). Therefore CTE teachers must understand the formal mathematical manipulation 
for math concepts embedded in their programs and avoid the use of methods that allow student to 
bypass abstract procedures and prevent their development of mathematical understanding. 

Contextual learning requires teachers to become facilitators for learning math content through 
experiences and real-life applications that are already familiar to students. When students have 
no prior experience on which to scaffold, teachers must provide authentic activities and 
exploration where students work in teams to apply math concepts and transfer gained 
understandings to new scenarios (CORD, 1999). In order to truly teach the numeracy embedded 
in their CTE area, teachers must be aware of the “relationships of academic curricula to personal, 
societal, and especially occupational life” (CORD, 1999, p. 60). 

Work-Based Learning 

In an effort to connect learning in CTE programs with the real-world workplace, beginning 
teachers will be called upon to integrate work-based learning. Historically, work itself was the 
only way of preparing for work — learning from father or mother by performing more and more 
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complex work tasks over time. In apprenticeship programs, both formal and informal, or through 
on-the-job-training, skills were learned as a means of entering a profession over a period of 
several years and through many stages. But these methods have their own limitations and 
political implications. Various forms of work-based learning have emerged to reform the 
practice, either as a complement to school-based or classroom learning: in co-op education 
programs established in 1906; in the continuation schools envisioned by the Smith-Hughes Act 
of 1917; in the internships that followed the work-experience programs of the 1970’s; and most 
recently in the School-to-Work Opportunities Act of 1994, which provided funding for programs 
that incorporated school-based learning, work-based learning and connecting activities to make 
the two consistent with each other (Grubb & Badway, 1995). 

Biggs, Hinton, and Duncan (1996) asserted that major changes in the educational infrastructure 
are necessary to support and build a quality work-preparation system for the 21 st century 
including work-based apprenticeship, career academies, school-based enterprises, and 
cooperative education. Lynch (2000) described a "new vision" for CTE in which the infusion of 
work-based learning contributes to mastery of industry standards. 

The use of work-based strategies in CTE may take many forms. Work-based learning may begin 
as early as middle school or early high school with job shadowing (Brown, 2003) and includes a 
range of activities that go well beyond the traditional practice of cooperative education. Such 
experiences are identified and described in Table 2B.1. 
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Table 2B.1 

Work-Based Learning Experiences 



Approach 


Characteristics 


Strengths 


Challenges 


Job Shadowing 


•Students observe people 
in a career of interest 
•Usually a few hours to 
one day in duration 


•Provides early 
experience in WBL 
•Low time 
commitment by 
teacher and student 


•Employer engagement 
•Limited opportunity to 
engage in real work 
tasks 

•Transportation for 
young students 


Career Fair 


A career awareness/ 
exploration event in 
which multiple business 
participants assemble at 
one particular site at the 
school to promote and 
share their respective 
occupations. Career fairs 
develop an awareness 
and understanding of the 
career fields. 


Career fairs can be 
simple or complex 
involving one or more 
classes at a time. 
Provides opportunity 
to explore many local 
career options in a 
short period of time. 


Coordination is 
required of teacher 
Limited information is 
gained by students 
No real hands-on 
application of 
knowledge. 


School-based 


•Students produce goods 


•Students apply 


•Locus can shift to 


Enterprises 


or services for sale to 
customers 


academic knowledge 
to work 

•Instructors maintain 
control of 

instructional activity 


production rather than 
instruction 

•Lack of understanding 
about how learning 
occurs in workplace 


Career 


•School within a school 


•Career focus may 


•scheduling conflicts 


Academies 


•Career field focus rather 
than specific job 
preparation 
•Integrated career and 
academic instruction 
•Includes necessary work 
skills 

•Employer involvement 


keep at-risk students 
in school 


•Requires involvement 
of business/industry 
•Requires collaboration 
and cooperation 
between academic and 
CTE instructors 



119 








Youth 

Apprenticeship 


•Work-experience in 
industry 

•Linkage between 
secondary and 
postsecondary education 
•Collaboration among 
groups 

•Requires modeling of 
tasks and coaching of 
students 


•Creates a learning 
situation that 
emphasizes skills & 
knowledge required in 
the workplace 


•Requires significant 
employer participation, 
workplaces are 
impacted 
•Potential conflict 
between student & 
employer needs 
•Potential legal issues 
for youth in certain 
industries 

•Requires collaboration 
& cooperation between 
schools and employers 


Clinical Work 


These experiences 


A clinical is a 


Requires extensive 


Experience 


usually take place in 
medical settings 
Students have 
opportunities to practice 
the skills they have 
learned in the classroom. 


structured practical 
application of 
previously studied 
theory. It is a 
combination of course 
work and part-time 
work experience. 


teacher supervision and 
cooperative from health 
facilities. 


Cooperative 


•Traditional CTE 


•Students have part- 


•Lack of coordination 


Education 


program 
•Written training 
agreements specify what 
students will learn & 
employer’s 
responsibilities 
•Students usually work in 
some combination of 
during and after school 


time job 

•Employers use as 
screening device for 
new employees 


between school & work 

experiences 

•Work may not related 

to competencies needed 

by student in CTE 

program 

•Work becomes more 
important than learning 
skills 


Internship 


•One-time short-term 
placement directly 
related to career goal 
•May be paid or unpaid 


•Students gain 
practical, first-hand 
knowledge of 
workplace 
•Form relationship 
with employer 


•May be little 
communication about 
work experience 
•Tend to occur at end of 
education program 



Simulations -Designed to comprise a 


•Feedback to student 


•May tend to appear 


more or less accurate 


is immediate 


artificial to students 


representation or model 


•The instructor 


•Does not have the 


of a real work experience 


controls the content 


same criticality as 


•Students interact in 


•Help develop critical 


employer-based 


much the same way they 


attitudes required for 


experiences 


would interact in the 


success in the 




workplace. 


workplace 
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Work 


•Designed to apply 


•Provides actual work 


•Requires instructor 


Experience 


learned skills in an actual 


experience for student 


time to observe 




work setting 


•Instructor can 


•Requires close 




•May last 9-18 weeks in 


observe student work 


coordination with 




last semester of CTE 


in actual work- setting 


employers 




course 




•Transportation to and 
from work- site may 
pose challenge 


Service 


Service learning 


Enhances already 


Requires cooperation 


Learning 


combines community 


existing curricula. It 


from community 




service with classroom 


offers teachers a tool 


Requires active 




instruction. It focuses on 


that complements 


involvement by teacher. 




critical, reflective 


learning and increases 






thinking as well as 


educational relevancy. 






personal and civic 


It combines academic 






responsibility. 


learning with service 
activities that are 
structured to address 
real needs in the 
community. 

Offers youth a chance 
to solve problems and 
become involved in 
the community. 





Sources. Table adapted from Rojewski (2002). Table structure and some content from Biggs, 
Hinton, and Duncan (1996). Additional content from Brown (2003), and Stone and Wonser 
(1990). 



Authentic work experiences should be designed to provide students with a vehicle for gaining a 
deeper understanding of the knowledge and skills they are learning and an organized manner in 
which these skills may be practiced and perfected (Stasz & Kaganoff, 1997). These skills must 
be provided with coaching and support from the instructor. Well-designed work-based learning 
activities have the potential of shaping students who are not only prepared for the work of the 
21 st century but who have also actually seen, touched, and experienced it. 
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Appendix 2C 



Concept Paper for Classroom Assessment Module 

“What classroom assessment practices should beginning CTE teachers know and be able to do?” 
The first part of this section defines and provides examples of two types of assessment: 
summative and formative. These two constructs are important to providing on-going feedback to 
improve student motivation and learning as well as to measure what students learn. The second 
part of the section focuses on developing assessment tools that provide feedback and measure 
student progress on CTE program content: CTE knowledge and skills; academic knowledge and 
skills; and the workplace readiness skills essential to the 21 st century workplace. These tools 
include teacher-made tests, rubrics and scoring guides, and portfolios. The next section discusses 
grading practices and how to report student progress. Finally, the context of standardized 
summative assessments of academic and CTE knowledge and skills that CTE teachers are 
expected to understand and interpret is addressed. 

Types of Assessment 

Assessment is the process of collecting, analyzing, and interpreting information to determine the 
degree to which students are meeting standards. There are a variety of reasons that teachers 
assess students, among them: to provide feedback to students during the learning process, to 
measure and report performance to students and to parents, to improve instruction, and to gauge 
readiness for further learning and the workplace. To the beginning CTE teacher, classroom 
assessment may seem like one of the simplest issues to be faced, but the way in which students 
are assessed, particularly the way in which students receive feedback during the learning process, 
is one of the most powerful tools for shaping student learning and motivation (Bangert-Drowns, 
Kulik & Kulik, 1991; Bangert-Drowns, Kulik, Kulik, & Morgan, 1991; Black & William, 1998; 
Crooks, 1988; Fuchs, 1986; Kuger & DeNisi, 1996; Natriello, 1987; all as cited in Marzano, 
2006). 

Consistently adhering to certain assessment practices throughout the learning process improves 
student achievement one-half to two standard deviations on high stakes tests with the largest 
gains being made by low achievers (Stiggins, 2005). Furthermore, the assessment methods used 
to provide feedback during learning can motivate students to persist in the learning process and 
reach learning goals (Stiggins, 2007). To achieve the maximum learning benefits, CTE teachers 
need to plan not only the summative assessments, those used at the end of the learning 
experience to determine what has been mastered, but formative assessments that provide on- 
going feedback throughout the learning process. To accomplish this goal, teachers will use a 
variety of assessment tools to determine how well students are meeting CTE and academic 
standards, as well as their progress toward competence in workplace skills for the 21 st century. 
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Summative Assessment 



Summative assessment refers to those assessments that occur after a learning event has 
concluded. There are many reasons for summative assessment. The one most often cited is to 
judge the success of a learning process at its completion (Cruickshank, Jenkins, & Metcalf, 

2009). Classroom teachers can apply the concept of summative assessment to a unit of study or 
to the end of a course. Summative assessments take a variety of forms, such as the traditional 
paper- and-pencil tests; problem-solving projects, performances or products; or research papers 
and oral presentations. These forms can also be combined. For instance, an end-of-course 
examination may include a written paper-and-pencil test component and a performance 
component. A multi- week unit of study may culminate with a project that includes an oral 
presentation and a research paper. Because CTE content includes a variety of knowledge and 
skills, teachers need a variety of assessment tools to measure what students have learned. Paper- 
and-pencil tests, rubrics, and portfolios assessment tools can all be used for summative 
assessment. 

Designing the summative assessment is one of the first steps in instructional planning once the 
CTE, academic, and workplace readiness standards have been identified. Traditionally, teachers 
have planned a summative assessment after planning a unit of study or course, but “backwards 
mapping,” or planning instructional learning activities after the summative assessment is 
developed, assures better alignment between the written curriculum (what is to be learned), the 
assessed or “tested” curriculum (what is measured), and the taught curriculum (what the teacher 
emphasizes through learning activities during and outside of the classroom). To develop 
assessment tools that are well aligned with standards, the Educational Testing Service (2004) 
suggests that teachers reflect on several questions: 

1. What are the CTE, academic, and workplace readiness standards students need to 
master in this course (or unit of study)? 

2. What knowledge and skills from these standards is most important? 

3. What kind of evidence would I need to determine whether or not students have 
mastered the most important knowledge and skills from these standards? 

4. What are the assessment tools I will need to capture and measure this evidence? 

5. How will I use this evidence in the future (e.g., to keep score, modify my own 
teaching, re-teach or use to improve lessons, chart the scores on the items or 
assessment over time)? 

6. How will I report this evidence and to whom? 

The answers to these questions provide information that can be used to develop a chart indicating 
the standards to be assessed, the type of evidence that is needed, and their relative weight on the 
assessment based on the importance of the content. This chart is called a table of specifications, a 
two-way chart that describes the standards to be covered by the assessment and the percentage of 
the total score that will be associated with each of the standards. The purposes of a table of 
specification are to ensure that fair and representative weight of the total assessment focuses on 
the key areas and weighs those different areas based on their importance. Tables of specification 
are designed based on the list of course standards, the amount of time spent on those topics, and 
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the emphasis given in textbooks, manuals, or curriculum guides. A template for a table of 
specifications is included in Table 2C.1. 

Table 2C.1 

Generalized Table of Specifications 



Technical Standards: Weight in Total Type of Evidence Assessment Tool 

Assessment Needed 

Standard 1: 

Standard 2: 

Standard 3: 

Academic Standards: 

Literacy Standard: 

Numeracy Standard: 

Science Standard: 

21 st Century Skills: 

Standard 1: 

Standard 2: 

Standard 3: 



The concept of a table of specifications can be applied to a whole course, a unit of study or a 
specific assessment such as one paper-and-pencil test or a rubric to assess a performance. With 
more experience and feedback, teachers may expand the table to include greater detail about the 
types of items, the cognitive levels for items, and various levels of acceptable performance and 
then classify items into various paper-and-pencil and authentic and performance assessment 
strategies. 

Another way to think about the composition of summative assessment items or tasks is to 
consider the level of cognitive complexity required to complete each item. The ideal assessment, 
according to Marzano (2006) consists of three types of items or tasks, each with a different level 
of cognitive complexity: 

• Type I items or tasks — address basic details and processes that are relatively easy for 
students 

• Type II items or tasks — address more complex ideas or processes 

• Type III items of tasks — require students to make inferences or applications that go 
beyond what was taught in class 

To construct Type I items the teacher would ask and answer the following question (Marzano, 
2006): “About this measurement topic, what are the basic details and processes students should 
understand or be able to do fairly easily if they were paying attention in class?” In an agriculture 
mechanics class, the teacher might provide a list of topics and ask the student to provide a brief 
explanation of each term. For example: 

A short weld used for temporarily holding metal in place is called a weld. 

A. spacer 
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B. temporary fusion 

C. tack 

D. temporary braze 

Although each term represents a very complex idea, the item may be answered with a general 
understanding of detail or process. Type I responses would require a basic physical 
demonstration of a skill or competency in an expected scenario or environment as taught in class. 

To construct Type II items, the information must go beyond simple recall of vocabulary items, 
demonstration, or characteristics. Type 2 items require students to produce information. To 
construct a Type II item, the teacher would ask: “About this measurement topic, what are the 
more complex ideas and processes students should understand and be able to do if they were 
paying attention in class?” (Marzano, 2006). For example, in a logistics course, the student 
would be asked to demonstrate the steps required to complete an order fulfillment process as 
follows: 

Using the facilities picking process, the participant will complete two order picks, 
entering the designated information/data on the forms provided. 

Type II items ask the student to demonstrate knowledge or skill in a more unpredictable or 
unusual setting that would require information outside classroom instruction. 

To construct a Type III item, the teacher would ask and answer the following question (Marzano, 
2006): “About this measurement topic, what inferences and applications might students be able 
to make even though they go beyond what was taught in class?” For example the teacher might 
ask, “Describe how you would select wood for framing a house in a very humid climate.” For 
Type III items, in a performance based assessment, the student would be asked to demonstrate 
the skill or process in very unpredictable situations. In an auto mechanics class, students may be 
asked to troubleshoot and diagnose a problem in a gas engine. What makes the process 
unpredictable is the way in which the instructor inserts “problems” or “bugs” that the student 
must diagnose. 

Developing a scoring scale. Given that an assessment contains the three types of items identified 
previously, Marzano (2006) suggests that it is relatively easy for a teacher to translate patterns of 
responses into scaled scores and then grades. Table 2C.2 provides an example of a simplified 
scoring scale 

To score the simplified scale, the teacher reads each students’ responses to each item marking the 
responses as correct or incorrect using a simple system like a “plus” sign for correctly answered 
questions, and a “zero or negative sign” for incorrectly answered questions and items which 
received no response. With time and experience a teacher may develop a more nuanced scoring 
scale with partial scores and additional conditions. This system allows the teacher to quickly 
grade assessments using the same criteria, which provides more uniformity of assessment across 
classes. 
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Table 2C.2 

Simplified Scoring Scale 



Topic Score on Scale 


Description of Place on Scale 


4.0 


In addition to Score 3.0 on performance, in-depth inferences 
and applications that go beyond what was taught 


3.0 


No major errors or omissions regarding any of the information, 
steps or processes that were taught in the classroom 


2.0 


No major errors or omissions in the simpler details and 
processes but major errors or omissions regarding the more 
complex steps or processes 


1.0 


With help, a partial understanding or some of the simpler details 
and processes and some of the more complex ideas and 
processes 


0.0 


Even with help, no understanding or skill was demonstrated 



Developing Paper-and-Pencil Tests as Summative Assessments 

Teacher-made tests remain the most common form of classroom assessment in schools today 
(Brookhart, 2004). Various studies estimate that K-12 teachers construct as many as 54 tests or 
assessments in a given course in a given year and spend nearly one-third of their time in 
development and maintaining assessment activities (Green & Mantz, 2002; Marso & Pigge, 
1991). They can be used to measure knowledge and understanding of any of the three types of 
content in CTE programs: CTE knowledge; academic knowledge; and knowledge of essential 
workplace skills such as 21st century skills. 

Because students will be exposed to the paper-and-pencil-type assessment format on college 
entrance and placement exams as well as on employer or industry-credential exams, it is 
important for students to regularly respond to paper-and-pencil test items as they learn CTE 
content. CTE teachers can access a variety of sources for sample test items such as textbook 
materials developed for teachers, item data bases, or released items from college readiness or 
employer/industry exams. Although teachers may draw test questions for a variety of sources, 
assembling items into a well-constructed teacher-made test has several advantages over 
standardized, textbook company-produced, or even local and state curriculum developer- 
produced exams. First, classroom teachers can key test questions to the very specific objectives 
and content they have taught, rather than to general content that “most” teachers teach for this 
subject” (Cruickshank et al., 2009). Secondly, they often can provide more detail and specificity 
then may be typical with standardized or company-produced “general” tests. Thirdly, they can be 
targeted specifically to the learners in this classroom and thus facilitate some accommodation for 
the varying and special needs of students. 

In spite of these advantages, teacher-made assessments are often poorly constructed and thus 
often provide inaccurate information (Cruickshank et al., 2009; Green & Mantz, 2002; Stiggins, 
2006). To improve test development, teachers should be cognizant of two universal concepts of 
good test construction: 
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• Validity - Do the test, test items, and assessment as a whole measure what they are 
supposed to measure; that is, the most important content or concepts that were taught and 
that the student must know? 

• Reliability - Do the test, test items, or assessment receive essentially the same score over 
time (i.e., consistently) and produce scores that are not affected much by chance? 

To design a paper-and-pencil test, even if using items from a resource or data base, CTE teachers 
can use a table of specifications described earlier to ensure that the number and type of items on 
the test correspond to the standards for the unit and their relative importance. Although the most 
common items are multiple choice questions and short answer or essay questions, paper-and- 
pencil tests can also include items in which the correct answer is matched to its description, 
statements which may be determined as true or false, and statements in which a blank must be 
filled in with an answer. Table 2C.3 illustrates the most common options of test items. Based on 
the listing of standards in the table of specifications, the teacher selects appropriate types of 
exam items. For example, if a standard involves analysis or application of information and is 
significantly important in the unit, the teacher may choose to create an essay question that is 25% 
or 30% of the total exam. If the standard involves remembering a sequence within a technical 
process, matching or short answer may be an appropriate type of question, with a small amount 
of the total exam devoted to that content. 

Table 2C.3 

Types of Paper-cincl-Pencil Test Items 



Multiple Choice Uses a stem which presents a problem or asks a 

question with 4-5 alternative responses. 

True-False Presents statement which asks students to 

identify it as true or false. 

Completion Presents items that require students to supply 

missing words from a statement or write a short 
phrase that answers a question posed in a stem. 

Matching Presents a list of stems, often called descriptors, 

in the left column and a list (usually longer) of 
alternatives in the right column. 

Short Answer or Essay Stem, question, or statements that require written 

response which can range from a few sentences 

or paragraphs to several pages. 



Formative Assessment 

Formative assessment begins at the onset and continues through the learning episode, providing 
feedback to students to help them correct misconceptions, understand mistakes, and reinforce 
their progress. It helps teachers determine how well their students are learning and how effective 
their instruction has been. Feedback through formative assessment strategies allows the teacher 
to adjust instruction and improve students’ performance before any final (i.e., summative) 
assessment of learning is conducted. The purpose is to use many different assessment methods to 
provide students, teachers, and parents with a continuing stream of evidence of student progress 
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in mastering the knowledge and skills required for course completion, mastery of standards, 
and/or certification or licensing. Table 2C.4 illustrates the variety of formative assessment 
techniques that CTE teachers can use to provide feedback to students on their progress in 
meeting CTE and academic standards. 

Table 2C.4 

Formative Assessment Techniques for CTE Classrooms 



Project Plans — Goals, Timelines, Checklists 


Drafts of Papers or Projects 


Checklists — Performances or Work 
Completed 


Quizzes 


Homework Practice 


Practice Presentations 


Journals, Learning Logs, or Documentation 
of Work-in-Progress 


Interactive Notes or Representations of 
Content such as Graphic Organizers 


Classroom Checks for Understanding such 
as Exit Slips, Hand Signs, One-Minute 
Papers describing what was learned and 
what questions still remain 


Classroom Questions, aloud or in writing 



The ultimate goal of formative assessment is to assess for learning (Stiggins, 2005) in contrast 
with the assessment of learning accomplished through summative methods (Wilen, Hutchison, & 
Bosse, 2008). According to Stiggins (2007), assessment for learning begins when teachers 
clearly describe for students the goals or standards for learning and show models of exemplary 
work. The students’ role is to strive to understand what success looks like and to use feedback 
from a variety of assessments to discover where they are now in relation to where they want to 
be and how to do better next time. Various researchers have outlined principles of assessment for 
learning to include: 

• Aligning learning goals with assessment methods and knowing achievement targets 
before instruction begins 

• Informing students about these targets and the nature of the assessments that will be used 
to measure them 

• Using a variety of assessment methods that accurately reflect student achievement 

• Involving students in the assessment process 

• Providing frequent feedback to learners that gives students a clear picture of their 
progress on learning goals and encourages them to improve 

• Modifying instruction based on assessments. (Stiggins, 2005, as cited in Wilen et al., 
2008) 

Developing Assessment Tools for CTE Content 

CTE teachers address three major types of content: CTE knowledge and skills; academic 
knowledge and skills; and the 21 st century skills essential knowledge and skills for workplace 
success. This wide variety of content requires a variety of assessment tools, each of which can be 
used in formative assessment during a unit of instruction or in summative assessment at the end 
of a unit, marking period, or course. In this section of the paper, we will provide a framework for 
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designing assessment tools and explain the use of three specific tools: rubrics that lend 
themselves well to the performance of skills; paper- and-pencil tests, which are typically used to 
measure knowledge and understanding of content; and portfolios that document student progress 
over time. 

Once this reflection has taken place, the teacher can: (a) choose from a multitude of assessment 
strategies and (b) design a table of specifications. Table 2C.5 provides examples of commonly 
used teacher-developed assessment strategies in the category of authentic assessment of 
performance. 

Table 2C.5 

Examples of Assessment Tools for Authentic or Performance 



Checklists 

Rating scales 

Rubrics 

Portfolios 

Other authentic tasks 



Written instrument that lists the specific elements deemed necessary 
for desirable performance, product, or presentation; teacher checks if 
each element is present. 

Lists the specific elements required of a product, performance, or 
presentation, but allows teacher to make a judgment of its quality. 
Illustrates with detail and description the various elements required 
for performance, product, or presentation. 

A collection of samples of student work that demonstrates 
accomplishment and achievement; may include video or audio tapes 
to illustrate. 

Depending on objectives and expectations, an interesting array of 
“other” assessments have been reported by teachers: field interviews, 
work samples, student peer or self written evaluations, observational 
notes, competitions, employment-based assessments. 



Using Rubrics to Assess Quality and Provide Feedback 

Though the original definition of rubric meant "marks in red" (Finson & Ormsbee, 1998, p. 80) 
or as Marzano Pickering, and McTighe (1993) stated, “the use of red earth to mark something of 
importance,” today a rubric is viewed as an assessment tool. Rubrics define the criteria by which 
a student's work should be judged. Therefore, the rubric becomes a scoring tool indicating "what 
counts" (Goodrich, 1997; Montgomery, 2000, as cited in Jackson & Larkin, 2002). Scores are 
awarded based on predetermined criteria set forth in the rubric. In the CTE classroom 
demonstration of performance is an important issue for students meeting standards. Performance 
tasks are the backbone of a performance assessment (Marzano et al., 1993) system and rubrics 
are an important tool for performance assessment. Rubrics can be used to measure CTE 
knowledge and skills, academic knowledge and skills used in context or as part of a 
performance, and essential workplace competencies such as 21 st -century skills and habits of 
success. They are well suited to the measurement of projects, presentations, performances, or 
problem-solving tasks. 

Descriptions of performance for each level are contained within the rubric and indicate 
gradations of quality from high to low. Depending on the type of rubric used, grades are awarded 
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by the total score only or by separate pieces being judged and then totaled into a final score. 
Teachers, parents, and students can view the work from both formative and summative 
assessment perspectives. The rubric scale usually runs from 1 to 4, with 4 describing the highest 
level of performance. Usually one level of a rubric is considered the acceptable level of 
performance. In a four-point rubric a 3 is considered an acceptable level of performance 
(Marzano, et al, 1993). More specifically, a scoring rubric consists of a fixed scale and a list of 
characteristics describing performance for each of the points on the scale. Table 2C.6 illustrates a 
rubric for a process or skill. Because one overall score is given for the process or skills, this is 
described as a holistic rubric. 

Table 2C.6 

General Rubric for a Process or Skill 



4 The student can perform the skill or process important to the industry standard 
with no significant errors with fluency. Additionally, the student understands the 
key features of the skill or process. 

3 The student can perform the skill or process important to the industry standard 
without making significant errors. 

2 The student makes some significant errors when performing the skill or process 
important to the industry standard but still accomplishes a rough estimation of 
the skill or process. 

1 The student makes so many errors in performing the skill or process important to 
the industry standard that he or she cannot actually perform the skill or process. 

0 No judgment can be made about the student’s ability to perform the skill or 

process. 



An alternate rubric form is an analytic rubric. This provides a description of levels of quality for 
different areas of criteria related to judge a product or performance. A particular product, such as 
a research paper, may have several areas of criteria including: substantive content, organization, 
grammar and conventions, and sources. Instead of just the one overall score that a holistic rubric 
yields, four different 1 through 4 scores would be judged for each of the criteria, which could be 
added together or weighted differently into a total score. Table 2C.7 provides an example of an 
analytic rubric for a brochure project. A CTE teacher may assign such a project to embed 
research, writing, and communication into CTE content. 

Table 2C.7 

Analytic Rubric for a Computer Technology Course Brochure Project 



CATEGORY 


(4) Excellent 


(3) Good 


(2) Almost 


(1) Not Yet 


Attractiveness & 

Organization 

(Organization) 


The brochure has 

exceptionally 

attractive 

formatting and 

well-organized 

information. 


The brochure has 
attractive 
formatting and 
well-organized 
information. 


The brochure has 

well-organized 

information. 


The brochure's 
formatting and 
organization of 
material are 
confusing to the 
reader. 


Content - Accuracy 


The brochure has 


The brochure has 


The brochure has 


The brochure has 
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(Ideas)/Process is 
Clearly Explained 


all of the required 
information (see 
checklist) and some 
additional 
information 


all of the 
required 
information (see 
checklist). 


most of the 
required 
information (see 
checklist). 


little of the 
required 
information (see 
checklist). 


Writing - 

Mechanics 

(Conventions) 


All of the writing is 
done in complete 
sentences. 
Capitalization and 
punctuation are 
correct throughout 
the brochure. 


Most of the 
writing is done in 
complete 
sentences. Most 
of the 

capitalization and 
punctuation are 
correct 

throughout the 
brochure. 


Some of the 
writing is done in 
complete 
sentences. Some 
of the 

capitalization and 
punctuation are 
correct 

throughout the 
brochure. 


Most of the 
writing is not 
done in complete 
sentences. Most 
of the 

capitalization and 
punctuation are 
not correct 
throughout the 
brochure. 


Graphics/Pictures 


The graphics go 
well with the text 
and there is a good 
mix of text and 
graphics. 


The graphics go 
well with the 
text, but there are 
so many that they 
distract from the 
text. 


The graphics go 
well with the 
text, but there are 
too few. 


The graphics do 
not go with the 
accompanying 
text or appear to 
be randomly 
chosen. 


Sources 


There are many 
citations from a 
variety of sources 
accurately listed on 
the brochure. 


There are some 
citations from a 
variety of sources 
accurately listed 
on the brochure. 


There are a few 
citations 
accurately listed 
on the brochure. 


Incomplete 
citations are listed 
on the brochure. 



Although rubrics can be time consuming to develop, they ultimately streamline the teachers’ 
assessment time by standardizing the expectations for quality performance. Students may submit 
higher-quality work for final assessment because the expectations are clearer and that can mean 
less time necessary for assessing the work. To save development time, teachers can access many 
ready-made examples of rubrics and templates for development that can be tailored to specific 
projects or assignments. These examples and templates can be found in various formats both on- 
line and in texts such as Marzano et al. (1993). When starting from scratch, Goodrich (1997) lists 
the following steps in developing a rubric: 



Step 1 : Look at Models 

Step 2: List the criteria 
Step 3: Articulate gradations 

Step 4: Practice on models 



Show students good and poor examples of student work 
for particular task. Help students identify characteristics 
of each. 

Use the characteristics to generate a discussion about 
what is considered quality work. 

Describe the best and worst level of quality on the of 
quality continuum and then fill in the middle levels of 
quality 

Have students use the rubric created in Step 2 and 3 to 
assess the examples of good and poor work in Step 1. 
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Step 5: Use self- and peer assessment Stop students occasionally as they are working on a 

task to have them and their peers assess the work. 

Step 6: Revise Encourage students to revise their work based on the 

feedback they receive in Step 5. 

Step 7: Use teacher assessment Assess students’ work by using the same rubric they 

have used to assess their own work. 

There are several major benefits to rubrics (Jackson & Larkin, 2002). First, the students know 
before beginning an assignment what the expectations of performance will be. The expectations 
may be assigned by the teacher or may be determined with the help of the students through class 
discussion. Rubrics are easily modified to reflect the expectations for special needs students. 
Ultimately, the students understand of the assessment criteria and their importance. The product 
or performance is strengthened when students are involved in using rubrics (Ward & Murray- 
Ward, 1999, as cited in Jackson & Larkin, 2002). 

Secondly, students can monitor their own progress as the assignment progresses. When students 
understand how to use rubrics, they can focus on what is considered important. Students become 
aware of the quality of work through judging their own and their peers' assignments against the 
standards set in the rubric. The rubric becomes a tool of formative assessment, to provide 
feedback and guide the learning experience or project. The rubric can also be used as a final 
checkpoint before turning in a project. 

Finally, the rubric focuses the final assessment for the project and provides clear justification for 
the grade assigned to the work. Once the final product is submitted, summative assessment is 
accomplished as the rubric is used to award a final grade. The rubric serves as a tool for 
communicating expectations for success in the final product among teachers, parents, and 
students. 

Portfolios 

Student portfolios are an effective way to document student learning and evaluate student 
progress over time. A student portfolio is a purposeful collection of student work that tells the 
story of the students’ efforts, progress, or achievement in relation to standards. When students 
actively participate in the selection of portfolio content and reflection on what they have learned, 
the portfolio is a powerful assessment tool (Meyer, Schuman, & Angello, 1990). Effective 
portfolios: 

• Are purposeful. Without purpose the portfolio is just a collection of student work. The 
portfolio can be used to show the process the student used to develop a project such 
as a pre-engineering project notebook or it can be used to demonstrate skills in 
building a truss or structural repair of an automobile to demonstrate end of course or 
program competence against industry standards. The purpose must be clear. 

• Require student reflection. Self-reflection of the student in selecting the content of the 
portfolio is the one thing that makes a portfolio instructional. 

• Outline criteria for judging merit. When a decision is made to include an item in a 
portfolio the decision is based on some criteria. The sharing of criteria is a great way 
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to recognize strong performance, provide instruction to students in identifying quality 
work, and using criteria to learn and improve performance. 

• Provide guidelines for selection of work. Guidelines provide direction for what to 
include in a portfolio and should be aligned with purpose and learning standards. 

• Involve the students in the selection of work. The true instructional value comes when 
students use criteria and self-reflection when making decision about what they want 
to show about themselves and why. 

Assembling a portfolio cannot be done without applying criteria to judge the quality of the work 
that is selected. The portfolio should never be seen as work “outside” the learning standards of 
the course (Arter & Spandel, 1991). If the portfolio is assembled based on criteria directly 
connected to the learning standards, the selection of the materials can paint a picture of student 
effort, growth and achievement. This is the very essence of assessment (Arter & Spandel, 1991). 
Thus the portfolio can be used to improve achievement, a value over and above merely 
assigning a grade or monitoring learning. 

Grading 

After collecting various assessment data throughout a course, CTE teachers will be expected to 
assign a grade that reflects each student’s performance. Grading is one of the areas of teaching 
responsibility where there is the biggest gap between research and practice (Guskey & Bailey, 
2001). Most teachers often reflect back on what they experienced as students and use grading 
strategies they perceive as fair, reasonable, and equitable (Guskey & Bailey, 2001). 
Consequently, grading practices are often based on personal experiences rather than lessons from 
research. If the central purpose of education is to promote student learning, then decisions about 
how to grade must be made with that overall purpose in mind. Guskey (2000) states: 

Grading requires careful planning, thoughtful judgment, and a clear focus on purpose, 
excellent communication skills, and an overriding concern for students. Such qualities are 
necessary to ensure grading policies and practices that provide high quality information 
on student learning in any standards-based learning environment. Grading and reporting 
should always be done in reference to specific learning criteria or standards, rather than 
an average or class curve grade. 

Although there are no right or wrong ways to design grades, Marzano (2000) suggests 
grades should provide information or feedback to students and parents and that the best 
reference for grading is content-specific learning goals or a criterion referenced approach. 
Teachers should identify what they want their students to learn, what evidence will be 
used to verify that learning, and what criteria will be used to judge the evidence. Teachers 
must clarify their standards and their grading criteria. Grades based on specific learning 
criteria and standards have direct meaning and serve well to communicate student 
progress in learning to the standards. 

To align grades with standards, CTE teachers can use the table of specifications described 
earlier in this section. The table for a course grade would include a list of the major 
standards for the course, including technical, academic, and workplace readiness. Based 
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on the relative importance of each of these types of standards in the course, these three 
categories of CTE content — technical knowledge and skills, academic knowledge and 
skills, and career readiness knowledge and skills— could be assigned a relative weight in 
the overall course grade. For example, technical standards and the assessments they 
measure might be 40% to 50% of the course grade; performance on academic knowledge 
and skills related to career and college readiness, 20% to 30% of the grade; and 
workplace readiness or 21 st -century skills such as teamwork, work ethic, and 
accountability, 20% to 30% of the grade. 

To provide effective feedback to students, teachers must develop a system for keeping track of 
the data related to each standard, so that both the teacher and the students can track progress 
throughout a marking period and over the course of the school year. Developing this system 
requires a careful consideration of the way in which assessment data are kept in a grade book. 
Traditionally, teachers keep grades in a two-dimensional format with the students names down 
one column and a series of scores on various assignments and assessments in the row 
corresponding to each student’s name. According to Marzano (2000), the most effective way to 
provide feedback through grading is by recording grade information in a three-dimensional 
format based on categories or topics related to the standards. For example, the assessment scores 
for a CTE unit may include tests, homework, projects, or performances. The assessment for a 
project might include a score for technical skills, for teamwork (a career readiness skill), and for 
academic knowledge (writing, for example). A grade book that is based on categories of the 
standards allows the teacher and student to track the progress of the student toward proficiency 
in each specific category measured rather than recording a single score for a project. 

This type of record-keeping goes beyond the typical point system in which each assignment or 
assessment is given a relative amount of points and the total points are averaged to determine the 
marking period grade. The point method makes sense if the teacher addresses only one topic, 
such as financial literacy in a family and consumer sciences class. What is wrong with the point 
system that has been used for nearly a hundred years? Nothing if all that is assessed are multiple 
choice, or other test items in which a correct/incorrect response is scored. The problem is that 
essay items, oral presentations, portfolios, and performance of skills or procedures do not lend 
themselves to a correct/incorrect grading scale. Thus, it becomes very difficult to accurately 
provide feedback to students on their progress specific to certain standards. 

To effectively keep track of grade data when multiple areas of standards are assessed requires the 
use of grading columns in a grade book based on topics related to the standards. The data in each 
category is recorded and then an overall total or score is determined in each category before it is 
calculated in the grade for the marking period. An example of a grade book based on categories 
of the standards is found in Table 2C.8. Based on the type and nature of each specific 
assessment, the part of the score relating to that category would be recorded in each column or 
left blank if that category was not part of the particular assessment. 
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Table 2C.8 

Grades Recorded by Categories of the Standards 





Assessments for the Marking Period (Examples): 

A. Quiz #1 

B. Homework Sept 18, Math Problems 

C. Performance Task 

D. Project 


Student’s 

Name 


Assessment 


Category 1: 
Technical 


Category 2: 
Academic 


Category 

3:21 st 

Century 

Skills 


Category 
4: All 
Aspects of 
an 

Industry 




A 


Score 


Score 


N/A 


N/A 




B 


N/A 


Score 


N/A 


N/A 




C 


Score 


N/A 


Score 


Score 




D 


Score 


Score 


Score 


Score 


Portion of 
Grade for 
Each 
Category 




Grade 


Grade 


Grade 


Grade 



The method of computation for grading based on categories is called the power law of learning. 

It is based upon the belief that learning increases as skills and knowledge are acquired and 
applied. It implies that learning is not equal from the beginning of the course to the end therefore 
grading should adjust to account for the change in learning from beginning to end (Marzano, 
2000 ). 

The simple or presumed simple, task of grading and setting up a grade book requires additional 
examination of local policy, educational philosophy as the instructor, and classroom application. 
Whatever method is selected for grading, the beginning teacher should be very familiar with 
local school district policies to make certain the grading scheme selected is within approved 
grading policies for the school district. If the end purpose of assessment and grading is to provide 
feedback to students and parents and to guide learning, the more applicable assessments and 
grading are to the actual learning taking place, the more powerful the result. 

In addition to determining a procedure for assigning a marking period grade based on standards, 
CTE teachers will also face day-to-day decisions about what to grade and how to assign scores 
for assignments and assessments. The difference between failure and the honor roll often 
depends on the day-to-day grading policies of the teacher (Reeves, 2008), rather than whether or 
not the students have actually met standards. For example, some common grading practices are 
so ineffective they can be termed “toxic” (Reeves, 2008): 

• Use of zeroes for missing work. Despite evidence that grading as punishment does not 
work (Guskey, 2000) and the mathematical flaw in using a zero on a 100-point scale 
(Reeves, 2004), many teachers use this policy, believing it will motivate students or 
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serve as punishment for bad behavior. Grades are ineffective as punishment and 
should not be used as a weapon for misbehavior. 

• Averaging all scores throughout the semester. This formula assumes that learning 
early in the semester is of the same level and quality as that later in the semester and 
often penalizes the student for early learning mistakes in the form of lower grades. 

• The “semester killer. ” The use of the single project, test, lab, paper or other 
assignment that makes or breaks a student and disregards learning that is 
accomplished throughout the semester. This practice puts 18 weeks of work at risk 
based on a project that consumes a small amount of time within the semester. 

CTE teachers will also need to think through their role in asking students to re-do work that is 
not to quality, and how the re-doing of that work will impact grading. Part of grading in CTE 
classrooms is recognizing that the goal is to prepare students for the workplace. One way to 
emphasize the importance of quality is to ask students to revise work that is of poor quality. Not 
accepting or grading work from students until it has reached a level of workplace quality 
provides an opportunity for the students to use feedback and understand the characteristics of 
quality work. In doing this the teacher must create an environment where students are not 
permitted to submit substandard work without being asked to revise (Scriffny, 2008). Through 
revising work, students will develop an accurate sense of what quality work looks like, which 
builds the self-sufficiency that is in demand by industry. 

Understanding the Larger Context of Student Assessment 

Schools, school districts, and states use many different methods to see how well individual 
students are doing. CTE teachers will benefit from an understanding of how schools, districts, 
and states use assessment data. Interpreting this data helps them understand the broader school 
context in which they work and prepares them to participate in building-level discussions about 
assessment data and how it is used in overall school improvement. At the local, state, national, 
and even international level, summative assessments yield information that is used to evaluate 
schools, programs, and/or a student’s standing relative to his or her peers or to a particular level 
of performance that may be necessary for graduation or entry into college. There has been wide- 
spread emphasis on these types of tests, also referred to as standardized tests, fueled by rising 
expectations for student learning. All 50 states require some form of summative, standardized 
testing to assure the public and taxpayers that students have learned that which the state values as 
important, typically in such academic content areas as math, science, language arts, and history. 
The results on the student’s test yields a “score” and these scores become a determinant for some 
future decision making. 

Summative assessments can be used to compare and rank students in school, make high-stakes 
decisions about grade or course failure, or to advance in higher level courses, such as advanced 
algebra or trig (Morrison, 2009). Nearly all states now use some standardized tests in core 
academic subjects to determine if students have mastered sufficient content matter to warrant 
receipt of a high school diploma. Another use of summative assessment is to compare the scores 
of one group against those of another or within a larger cohort, say 1 1 th graders on math in XYZ 
High School compared with those in the district as a whole and those in the state. 
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The score on standardized tests may also be a factor for admission to a particular college or 
university, and some employers examine the results of student’s scores on standardized tests to 
make decisions about employment or placement within their firm. The mantra being currently 
touted relative to high school graduation is to use testing to determine students’ college or career 
readiness. In fact, and as a condition for receipt of federal funding via Perkins IV, states are 
required to assess students’ attainment of challenging technical skills proficiencies that are 
aligned with postsecondary program or industry-recognized standards. CTE students are 
expected to have mastered state -recognized academic knowledge and/or industry-recognized and 
technical skills that “prepare them to enroll in a community college, four- year college, or 
university; pursue advanced career training in a public or private proprietary institution; enter an 
apprenticeship program; obtain employment; or enlist in the military’’ (Derner, Klein, & Hilber, 

2008, p. 1). 

It is strongly recommended that new teachers examine the testing systems being used in their 
school district and in their state to assess or measure both students’ readiness for college and 
readiness for careers. There will be plenty of information on the websites of the various testing 
companies or agencies to inform teachers about the academic knowledge and/or technical skills 
being assessed. The purposes for and uses of the exam(s) will usually be described. Often sample 
questions are provided. A perusal of this information should inform the new teacher of the 
academic and technical expectations that underpin the assessment. The teacher can then 
incorporate into his or her lessons for CTE students the technical knowledge and skills that are 
deemed important to licensing, certifying, or credentialing in the occupational area, as well as the 
related academic content to underpin the technical skills. The following two sections contain 
information about the nature of tests that are currently being used throughout the country to 
assess high school students’ readiness for college and readiness for careers. 

College readiness. To determine college readiness, most schools in most states examine 
summative data from one or more of four primary sources: students’ completion of all course 
requirements in that state’s college-prep high school curriculum; their cumulative grade point 
average; references and/or a description of school and community activities; and their scores on 
standardized tests from one of two national testing agencies, either the ACT or the College 
Board’s SAT. The ACT test assesses high school students' general educational development and 
their ability to complete college-level work. The multiple-choice tests cover four skill areas: 
English, mathematics, reading, and science. An optional writing test measures skill in planning 
and writing a short essay. The SAT assesses critical thinking and problem solving skills in three 
areas: critical reading, mathematics, and writing. 

There are some postsecondary institutions, primarily two-year technical and community colleges 
that use college-readiness summative tests to assess students’ potential for college, place them in 
appropriate college-level courses, and/or connect them to resources such as developmental 
studies or remedial programs, should they need assistance to enhance their success with college- 
level studies. Examples of three such tests or testing systems are as follows: 

• CLEP (College Level Examination Program, administered by the College Board) - a 
series of exams for testing examinees’ college-level knowledge acquired through course 
work, independent study, cultural pursuits, travel, special interests, military service, and 
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professional development. CLEP exams are available in business, composition and 
literature, foreign languages, history and social studies, science, and mathematics. 



• COMPASS (administered by ACT) - a computer-adaptive college placement exam that 
evaluates a potential college students’ skill levels in reading, writing, math, and English 
as a second language. 

• Wonderlic - a 12-minute, 50-question assessment often used to assess the aptitude of 
prospective students for postsecondary career and technical programs and specialized job 
training programs. 

Career readiness. The assessment of career readiness is not as clear cut nor does it have the 
history of research, development, and wide-spread use in schools comparable to the college- 
readiness assessments identified with the College Board or ACT. Perkins IV requires that 
technical skill measures be valid and reliable and focused on “career and technical skill 
proficiencies, including student achievement on technical assessments, that are aligned with 
industry-recognized standards, if available and appropriate” (as cited by Hyslop, 2009, p. 27). 
States are being encouraged by the federal government to report technical skill measures for CTE 
concentrators by using a state-established technical skill assessment or with a third-party 
industry-recognized certification or credentialing exam. There are a few states such as Arkansas, 
North Carolina, Oklahoma, Utah, and West Virginia that have developed and are using end-of- 
course and end-of-program technical assessments, but most states intend to rely on third-party or 
industry-recognized certification (Demer et al., 2008). 

In a review of state websites and publications about measures of career readiness for career and 
technical education and other students, there appears to be three major sources of technical skills 
assessment: certification exams, industry certification, and state occupational or professional 
licensing. Some of these may only be available or possible for CTE students in postsecondary 
programs and/or for those who are in secondary-postsecondary articulated or dual enrollments 
programs; that is, they may require graduation from high school before a certificate or license is 
awarded. The following is a brief description of some of the types of summative assessments that 
are available to measure career readiness: 

• Certification exams. If passed successfully, certification exams are used to recognize 
an individual’s attainment of validated skills. The exams may include off-the-shelf 
technical skills assessments or customized written tests or performance assessments 
used to determine whether test takers have achieved a set of generic work readiness 
skills or more specifically-defined technical skills within a discrete occupational area. 

• Industry credentialing. Typically defined as completion of an industry certification 
program with any number of courses (i.e., 2-4) leading to endorsement for 
employment in a specific occupation such as certified nursing assistant (CNA) or in 
food service operations (ProStart). In addition to successful completion of the 
coursework, students will usually complete an assessment from an agency such as the 
NOCTI or the National Restaurant Association. 
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• Occupational Licensing. Used to certify that an individual within a state is qualified 
to engage in a given trade or profession. Upon submission of appropriate 
documentation, the individual is issued a license that certifies him or her to practice 
that occupation or profession within that state. Typically a written and/or performance 
assessment is required before a license is granted. 

Examples of organizations that have engaged in research and development of skill assessments 
for use in career and technical education programs are included in Table 2C.9. Examples of the 
occupational skills for which they provide summative assessments (e.g., written and/or 
performance or job skill tests) are also included. 

Table 2C.9 

Organizations with Standardized Assessments 



Organization 


Examples of Occupational Skills Assessed 


Adobe Systems Incorporated 


Adobe certified associate 


Air Conditioning and Refrigeration 
Institute 


Residential air conditioning and heating 
certification 


American Design Drafting 
Association 


Drafting 


American Hotel and Lodging 
Education Institute 


All levels in the hospitality industry, ranging 
from front desk representative and restaurant 
server to certified hotel administrator 


American Welding Society 


Entry-Level welder 


ASE/AES/SkillsUSA Partnership 


Technician testing in automobile, truck, or 
collision repair 


A*S*K Business Institute 


Finance, fundamental marketing concepts, 
fundamental business concepts, 
entrepreneurship 


Brainbench 


Network administration, systems 
administration, software development, Web 
administration, desktop publishing, AutoCAD 


Cisco Systems 


Wireless LANs, Java programming, network 
associate, PC hardware and software technician 


CompTIA 


A+ certification 


Delmar Thompson Leaming/Home 
Builders Institute 


Residential construction, electrical, house 
wiring 


Dental Assisting National Board, Inc. 


Certified dental assistant 


Department of Health, Office of 
Emergency Medical Services 


EMS first responder, emergency medical 
technician 


Electronics Technicians Association 


Computer service technician, electronics 
technician, satellite dish installer, data cabling 
installer 


Electronics Technicians Association, 
International 


Student electronics technician 


Environmental Protection Agency 


EPA technician 
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Graphic Arts Education and Research 
Foundation 


PrintED certification 


HVAC Excellence 


Heating, electrical, air conditioning technology 


Manufacturing Skill Standards 
Council 


Certified production technician 


Microsoft 


Certified professional, office specialist, 
certified application specialist, systems 
administrator 


National Center for Construction 
Education & Research 


Carpentry, construction, masonry 


National Center for Competency 
Testing 


Beginning and advanced bookkeepers; several 
specialized medical fields such as medical 
assistant, medical office assistant, and ECG, 
phlebotomy, patient care, surgical, and 
pharmacy technicians; medical insurance and 
coding specialists; and postsecondary 
instructors 


National Institute for Automotive 
Service Excellence 


Automotive technician 


National Institute for Metalworking 
Skills 


Machining 


National Restaurant Association 


Sponsors a two-year curriculum, called 
ProStart, for high school CTE students in food 
service operations and management, 
mentoring, and a national certificate of 
achievement for students who successfully 
complete the program and Year 1 and 2 exams 


National Retail Federation 


Customer service, sales 


NOCTI - National Occupational 
Competency Testing Institute 


Written and technical performance skill 
assessments in over 170 occupational 
categories or jobs ranging from agricultural 
mechanics, banking, construction, dental 
assisting, electronics, floriculture, graphic 
communications technology, and on through 
the alphabetical listing up to welding and 
workplace readiness 


Novell 


Certified Novell administrator 


SkillsUSA 


Automated manufacturing technology, 
advertising design, photography 


WorkKeys, from ACT 


Foundational skills assessments measure 
different applied job skills in areas of 
communication such as business writing, 
listening, reading for information, and writing; 
problem-solving in applied technology, applied 
math, locating information, and observation; 
and in the interpersonal skill area of teamwork. 
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ACT also provides a national career readiness 
certificate for those who successfully complete 
exams on locating information, applied 
mathematics, and reading for information 
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Appendix 2D 



Concept Paper for Classroom Management Module 

“What classroom management practices should beginning CTE teachers know and be able to 
do?” This section begins with an explanation of the importance of effective classroom 
management and its relationship to student achievement, as well as the way in which classroom 
management is effected by teachers’ beliefs. The essential role of preventive practices in 
effective management is described followed by a discussion of the importance of understanding 
school policies and legal issues. The second part of this section outlines ways to implement a 
classroom management plan in CTE classrooms beginning with the first weeks of school then 
moving on to rules and procedures, disciplinary interventions, and ways to address the particular 
challenges of lab management. Finally, ways to support student success are outlined, specifically 
communicating with parents, providing extra help, implementing career and technical student 
organizations, and serving as a student adviser. 

Understanding the Importance of Classroom Management 
Classroom Management and Student Achievement 

A strong research base in education supports the conclusion that it is the intersection of high- 
quality instruction with an effective teacher that is a critical factor to enhance student 
achievement. Further, more than 30 years of research indicate that classroom management is a 
critical component of teacher effectiveness. Solid, student-engaging instructional strategies 
coupled with effective management of student behavior are the two critical sides of the 
classroom management coin (Phi Delta Kappa, 2006b). 

Classroom management, in its most basic form, can be defined as the provisions and procedures 
necessary to create and maintain a classroom community in which teaching and learning can 
occur (Cruickshank et al., 2009). This definition suggests classroom management extends far 
beyond merely making and enforcing rules to the quality of the student-teacher relationship 
itself. From a meta- analysis of 100 studies, Marzano, Marzano, and Pickering (2003) reported 
that teachers who had high-quality relationships with their students had 31% fewer discipline 
problems, rule violations, and related problems over a year’s time than did teachers who did not 
have high-quality relationships. The effect of the teacher-student relationship was not just on 
student behavior. Significantly greater rates of student engagement and higher achievement 
scores occurred in classes where effective classroom management techniques were used 
compared to classes without effective classroom management. Marzano and Marzano (2003) 
pointed out that effective teacher-student relationships are characterized by three components: 

1. exhibiting appropriate levels of dominance, defined as establishing clear behavior 
expectations and learning goals and exhibiting assertive behavior; 

2. exhibiting appropriate levels of cooperation characterized by a concern for the needs 
and opinions of others where the teacher and student work as a team, and 

3. being aware of high- needs students and having a repertoire of techniques to meet 
their needs, (p. 64) 
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For many new teachers, the effective management of the classroom and related disciplining of 
students looms as a very challenging, sometimes daunting concern. A survey conducted by the 
U.S. Department of Education concluded that less than 20% of first- year teachers consider 
themselves well prepared to deal with classroom management and discipline. Teachers who 
come to the profession through alternative licensing feel even less prepared than their 
traditionally prepared peers for the core tasks of teaching, such as meeting the needs of diverse 
learners, and constructing a positive learning environment (Darling-Hammond et al., 2002). 
Further, a Public Agenda survey concluded that 30% of teachers do not consider many other 
teachers as effective classroom mangers (Phi Delta Kappa, 2006a). Fortunately, teachers can be 
taught to be effective classroom managers. According to Marzano et al. (2003), good classroom 
managers are those teachers who understand and use specific research-based techniques that 
direct their behavior which, in turn, changes the behavior of students to greater engagement with 
the content and activities and results in improved achievement. 

Teachers’ Beliefs about Student Learning 

Any discussion of classroom management needs to begin with a self-assessment of fundamental 
beliefs about student learning. When teachers believe that all children can learn and that it is 
their job to ensure that they do so, the students in their classes achieve at higher than expected 
levels (Corbett, Wilson, & Williams, 2002). Beginning teachers need to spend some time 
thinking about their own views about students and how students learn and can be taught to learn 
more, their perspectives and acceptance of the diversity that typically exists in public school 
classrooms, and how to establish a classroom environment that maximizes learning and 
minimizes distractions. 

Saphier (2005) described three messages that seem to be part of the day-to-day and minute-to- 
minute classroom interactions when teachers hold the belief that effort makes the difference in 
learning: (a) “This is important.” The content, knowledge and skills students are learning will be 
important to them now and in the future; 2) “You can do it.” All students have the intellectual 
capacity to do rigorous work and meet high standards; and (c) “I won’t give up on you” (p. 90). 
If students do not succeed the first time, the teacher will provide extra time and support and help 
the student persist to mastery of the content. Table 2D.1 contrasts these effort-based beliefs and 
practices with the actions teachers take when they hold the more traditional (and incorrect) view 
that student success is based solely on ability. 
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Table 2D.1 

Ability-Based vs. Effort-Based Beliefs and Practices 



Ability-Based Beliefs and Practices Effort-Based Beliefs and Practices 



Students who are perceived to have the 
highest ability receive the highest marks and 
are given the most challenging assignments. 
Students perceived with less ability receive 
less challenging assignments and lower 
expectations. Any academic deficiencies 
students have are attributed to low ability. 

Time is the constant in learning, so students 
that fail to finish assignments, score well on 
tests, or learn key concepts by the “due 
dates” receive failing marks with no second 
chances. 

Students have the responsibility to motivate 
themselves to learn. If they do not believe 
they can do well in school, they probably 
won’t. 

Feedback to students is limited, often 
occurring only in the form of a numerical or 
letter grade. 

Teachers assume that students have the skills 
needed to be successful students by the time 
they get to high school. 



Effort makes a difference. Academic 
achievement can be grown. It is not how smart 
the student is, but how hard he or she works 
that determines success. All students are held to 
high expectations and offered opportunities to 
complete challenging assignments. 

Students learn at different rates and may not 
reach proficiency at the same time. A mistake is 
not an inability to perform, but a learning 
opportunity. For that reason, students may re-do 
work and retake tests. 

Students can be motivated to come to the belief 
that their effort is worthwhile, even if they do 
not believe it at the time they enter school. 

Students are provided with extensive and 
specific feedback through the learning process 
to make corrections in their understanding and 
continue to learn. 

Teachers explicitly teach students how to exert 
effective efforts in learning — study skills, time 
management, problem solving, and note-taking. 



These effort-based beliefs and the actions that are based on them have been found to motivate 
students who have typically struggled in school and close achievement gaps (Saphier, 2005). 
Fundamentally, the teachers who hold these beliefs and seem to get the best from their students, 
course after course, year after year, establish a classroom and lab environment that motivates and 
encourages all students to do their best; get to know their students — their learning styles, cultural 
backgrounds, career interests, education plans and aspirations, perspectives, and development 
needs; and manage their class and laboratories so that time is efficiently used, important tasks are 
given sufficient attention, transitions are well planned, and interruptions are kept to a minimum 
(Kellough & Carjuzaa, 2009; Kellough & Kellough, 2007). Thus, the effective CTE teacher 
plans lessons based on knowledge of the diverse needs of all of the students, with the conviction 
that all will learn the content and skills that are expected, and organizes and plans the classroom 
and activities accordingly. Planning includes attention to prerequisites (i.e., ensuring that 
students have the knowledge and skill necessary to move on), pacing, transitions, monitoring 
activities, systematic reviews, and feedback. 
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Planning also refers to the teacher giving some thought to what some authors refer to as the silent 
(or hidden) curriculum, referring to important behaviors and skills that are expected of students 
in schools and of all of us at workplaces and in communities. This is the nonacademic 
curriculum, sometimes referred to as competencies in the affective domain, and it needs to be 
modeled and implied in the management procedures used by the teacher. This might include, for 
example, competencies such as “respect for others,” “listening while others are responding,” 
“courtesy,” and “following directions.” For CTE programs, the silent curriculum is best defined 
through the life skills outlined in workplace readiness standards such as the 21 st Century Skills 
(Partnership for the 21 st Century, 2008). These student outcomes are the skills, knowledge, and 
expertise students should master to succeed in work and life in the 21 st century. The “Life and 
Career Skills” category of the 21 st Century Skills framework outlines specific essential outcomes 
to be targeted and taught through effective classroom management, such as 

• Flexibility and Adaptability: Deal positively with praise, setbacks and criticism 

• Work Independently: Monitor, define, prioritize and complete tasks without direct 
oversight 

• Interact Effectively with Others: Conduct yourself in a respectable, professional 
manner 

• Guide and Lead Others: Use interpersonal and problem-solving skills to guide and 
lead others toward a goal 

• Collaborate with Others: Demonstrate ability to work effectively and respectfully 
with diverse teams. (Partnership for the 21 st Century, 2008, pp. 4-7) 

Beginning teachers can start by making a list of the skills they expect from students, posting 
them in a prominent place in the classroom or lab, and then planning to teach those competencies 
within the context of academics and technical skills and discuss them frequently with students. 
Introducing these skills to students by discussing them as concepts and breaking them down into 
specific descriptions of what they look like when practiced helps students gain a clearer 
understanding of the behavior they are trying to master. For example, collaborating with others 
involves specific abilities to manage emotions, understand conflict, and know the difference 
between being aggressive and assertive. Once the teacher has articulated the desired behaviors to 
students, modeling, practicing, and assessing those skills needs to become an integral part of 
day-to-day classroom practice. Contextualizing these complex competencies required or 
expected in the workplace helps to reinforce their importance to the total development of the 
student. 

Teachers who clarify, model, and assess the essential workplace knowledge and skills of the 
“silent” curriculum effectively, implement a well-planned and validated sequence of instruction, 
and work well with students add to the credibility of the profession and have more positive 
feelings about themselves as persons and as educators. Several authors have pointed out that 
effective teachers rarely have difficulty coping. Hosford (1984) summed it up well: 

Effective teachers manage well. Coping is rarely an issue. The students are so busy at 

task-related activities, following sensible routines, and striving toward clearly understood 

objectives that situations with which teachers must “cope” seldom have an opportunity to 
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arise. Through management skills, superior teachers achieve what has commonly been 
labeled “preventive discipline” in the professional literature. They are not automatically 
superior teachers. They plan, worry, and work hard. I have never known superior teachers 
who “took it easy.” But the secret to their success — what sets them above the good 
teachers who also work, plan, and worry — is their process of management. They have 
learned (and firmly believe) that process affects product; that how they manage their 
classroom significantly affects the climate, motivational and goal achievement in their 
classrooms. In short, their knowledge base includes a thoughtful understanding of the 
importance of the Silent Curriculum, (p. 145) 

Regardless of how well teachers plan lessons and activities and given the nature of adolescents 
(and some adults), in all likelihood teachers will still encounter issues with classroom 
management. The key is to try to plan ahead to keep serious disruptions and behavioral problems 
to a minimum and avoid potential catastrophes. 

Planning an Inviting, Personalized Environment in the CTE Classroom 

The old adage, “an ounce of prevention is worth a pound of cure,” describes well the prevailing 
view on minimizing classroom problems. Teachers need to carefully consider the classroom 
environment they wish to create. Early studies in education showed clearly that the classroom 
environment has a powerful impact on students’ behavior, learning, and motivation (Cruickshank 
et al., 2009; Lewin, Lippitt, & White, 1939). There are essentially two sides to consider: the 
physical environment and the psychological environment. 

Physical environment. Teachers can begin by planning and arranging the physical environment 
well before the students arrive. To set the stage for preparing students for the 21 st century 
workplace, a CTE classroom should resemble an attractive, well-organized, and inviting 
workplace comparable to the career field. Students need to see the room or lab as a safe, 
comfortable place for positive social and academic experiences. Studies have shown that an 
attractive, well-organized classroom environment leads to more positive attitudes, better grades, 
and more receptive students. Conversely, unattractive classrooms have been linked to frequent 
absenteeism, discomfort, fatigue, and complaints from both parents and students (Cruickshank et 
al., 2009). 

For CTE classrooms and labs, equipment and computers should be up-to-date and neatly 
arranged; tools and supplies should be available and easy to access. Textbooks, manuals and 
workbooks, and other resources should be easy to identify and locate. Chalkboards, bulletin 
boards, display units, motivational and other posters, and other identifiers should be fresh and 
attractive and changed periodically, say at least once a month. State-of-the-art technology (e.g., 
SmartBoard, LCD panels, teacher computer station) should be available and prominent. All of 
this should be encased in a room in which the teacher uses color, light, temperature, soft tones, 
flooring, and other aesthetic elements to create an attractive, interesting classroom or lab. 

The physical arrangement should allow the teacher fluid access to and around all stations, seats, 
cubicles, or tables at which the students will work and study. Seating arrangements, too, need to 
be planned in advance. Ideally, the desks, tables, or equipment will allow the teacher (with the 
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help of students) to align seating arrangements with the methods of instruction. That is, 
classroom equipment provides the capability for students to work in small groups, in a circle for 
larger-group discussions, at independent study, or designed for large-group interaction such as 
tables in a “u” shape or lined up occasionally in rows. No single arrangement is ideal for all 
classes, learning situations, or individuals. In general, studies have shown that wide walkways 
are best to accommodate teacher and student movement. Wide walkways are especially 
imperative for some students with special needs, such as the wheelchair-bound and ADHD 
students. 

Psychological environment — personalization . . As Dr. James Comer (2001) has suggested, that 
no significant learning occurs without a significant relationship. There is an emerging consensus 
in school reform literature that a personalized learning environment contributes to student 
success (Annenberg Institute for School Reform, 1994; Darling-Hammond, 1994; both as cited 
in Klem & Connell, 2004). “Personalization” means that the teacher strives to know his or her 
students well, respects them, and supports them. In CTE classrooms, the teacher also represents 
an early contact the student has with a career field of interest. This provides a unique opportunity 
for the teacher to serve as a career mentor and be a powerful motivational factor in a students’ 
success in school and potentially in the workplace. 

As a workplace mentor, the teacher can model the tone and manner that resembles a well-run 
business by taking actions that reflect those of a positive, supportive, manager on site to teach 
and train business associates. Much like the workplaces of today, research shows that many 
disciplinary problems can be prevented with better teacher-student relationships. This doesn’t 
mean being overly friendly or a big brother or sister to students. It means showing concern for 
the needs and opinions of students and showing them that you want to work with them to 
accomplish learning goals, much as a manager does at a job site when training an employee. 

Students are, however, not yet adults in a real-world workplace. Although it is important to give 
them an understanding of how a real-world workplace operates, personalization for a secondary 
CTE teacher also means being aware of the students’ developmental needs, interests, learning 
styles, and hopes and dreams. It means relating to students by listening to them, making eye 
contact, and giving them complete attention during interactions (Canter, 1996). To achieve a 
personalized classroom environment, teachers need to purposefully seek information from their 
students in the first few weeks of school and use that information to plan instruction. This 
information includes the students’ career goals, learning preferences, course schedule, part-time 
work commitments, school activities, and family members’ names and contact information. 

Personalization also includes being culturally aware and responsive to students. A culturally 
responsive teacher respects diversity and celebrates students’ cultural backgrounds rather than 
attempting to override or negate them (Lieber, 2002). Teachers should make an effort to 
welcome diversity by learning about their students’ cultures, highlighting the accomplishments 
of people from different cultures, observing and learning about students’ cultural experiences, 
and considering students’ cultures and language skills in developing lessons. For example, 
effective teachers speak respectfully of students and are aware of words and actions that the 
students, especially those from cultures different than those of the teacher, may find offensive or 
disrespectful. 
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There are many ways for teachers to show they care about students. Greeting students each day 
with a smile as they come into the classroom and lab, much as good employers greet their 
employees at work, allows teachers to connect with students and to notice students who may be 
unready to focus on the learning activities ahead. Once teachers get to know students as 
individuals, they can recognize special events or celebrations in their lives; for example, a 
promotion at work, a birthday, an accomplishment at school, a job well done on a project or 
assignment in another class, etc. When students feel that their teachers care about them, they are 
more intrinsically motivated to learn and to assume responsibilities in the classroom (Grolnick, 
Ryan, & Deci, 1991). 

Understanding School Policy and Legal Issues 

Before the students arrive for the first day of school, it is incumbent upon the new teacher to 
learn of any laws and school policies that govern issues that relate to student management in the 
school. Most schools or school districts have policies in place that outline the steps or procedures 
that teachers and school leaders are to follow related to student conduct and violations that are 
either against the law or school policy. Individual classroom policies, rules, or procedures must 
be consistent with those established by law and by the district’s school board or the school 
council. Laws and policies are usually spelled out in a faculty handbook or student code of 
conduct booklet. It is expected that teachers and students know, understand, and abide by these 
policies and rules. 

The specificity in the manuals will depend on state and local laws and school district and 
individual school policies. There probably will be “rules,” based on laws that govern crimes such 
as the use of illicit drugs or tobacco by minors, bringing guns and knives onto school property, 
terrorist threats, and theft. The manuals may also inform teachers and students about their legal 
rights, such as the right to refuse police searches of their pockets or purses, cars, lockers, and 
book bags without probable cause and the right not to be questioned by police without an 
attorney present. There may also be policies, procedures, and punishments that are to be 
followed regarding many other student conduct violations. They may, for example, cover 
absences, tardiness, leaving school early, bullying, harassment, disrespect to adults or peers, 
fighting, extreme rudeness, obscene language, breach of security, use of cell phones and 
electronic devices, etc. Regardless, it is important that teachers be schooled themselves in the 
school’s policies on student conduct and understand the procedures that are to be followed 
should a violation occur. However, when reviewing rules with students in class, the focus should 
be on the few rules that are specific to the classroom, based on curriculum and workplace 
expectations, rather than the myriad of policies of the school or district as a whole. 

Implementing an Effective Classroom Management Plan for CTE 
Before School Begins 

Beginning teachers can use the time before the new school year starts to get a jump on their 
classroom management plans. By thinking through the importance of creating a positive learning 
environment, getting set with the curriculum and lesson plans, acquiring knowledge of school 
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district and school policies, establishing classroom procedures and “rules,” and getting the 
classroom or lab in shape, new teachers are well on their way to start the year. Lieber (2002) 
suggested several specific ideas for teachers to do before the school year begins that can help 
create a collaborative, personalized learning environment. Teachers can: 

• Write letters to parents or guardians to mail before school, post on the school or class 
website, or to send home with students on the first day of school. The letter should be 
one page and describe what the course is about, the teacher’s hopes and dreams for 
students, assignment and homework expectations, what might be challenging about 
the class, and what steps students can take if they are having difficulty meeting the 
requirements. This is the place to emphasize that effort, attitude and participation 
really count. 

• Design an assessment and record-keeping system that is standards and student 
friendly. The teachers should consider the kinds of assessments to use, particularly 
those for important to overall success in the workplace such as literacy, numeracy, 
teamwork, problem-solving and other 21st century skills. 

• Stock up on basic learning tools and supplies. Active engagement strategies require 
supplies such as newsprint, markers, tape, note cards, folders, boxes, post-it notes. 
Tools such as a timer and digital camera may also be helpful. 

• Give some thought to the arrangement of chairs and desks so that it is easy for all 
students to learn each other’s names. 

• Make the classroom “ours.” Teachers can post goals, routines, and procedures around 
the room, make a place for posting student questions, the daily agenda, and discipline 
rules. A space to keep homework assignments and study guides is also helpful. 
Finally, teachers and students will benefit from wall space devoted to displaying 
student work. (pp. 207-224) 

Rules and Procedures 

Before the school year begins, teachers can also develop the rules and procedures that will 
prevent major disruptions and poor behavior. Through a meta-analysis of classroom management 
research, Marzano et al. (2003) concluded that classroom rules and procedures significantly 
minimize classroom disruptions. They noted the following general categories in which teachers 
typically use rules and procedures: 

• General expectations for behavior 

• Procedures for beginning and ending the class period 

• Transitions and interruptions 

• Materials and equipment 

• Group work 

• Seatwork and teacher-led activities (p. 18) 

Although it is not necessary to develop rules and procedures in all of these categories, the first 
category is frequently one that new teachers include in a classroom management plan. These 
general classroom rules should guide conduct and behavior in a variety of contexts. Possible 
general expectations for behavior include: 
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• Respect others 

• Respect property 

• Bring materials to class 

• Be in assigned seat at the beginning of class 

In addition to sharing behavior rules, teachers have the option of explaining the concepts 
underlying these rules. Concepts such as: 

• Unkind words or actions will not be tolerated (Examples of put-downs or 
disrespectful behavior may be provided ). 

• Behavior is respectful when it does not create a disruption or endanger you or 
anyone else in the classroom 

The instructor may also communicate their own philosophy and anticipated behaviors. Examples 
might include: 

• I will react without anger or haste to problem situations. 

• I will provide consequences for inappropriate behavior that support your making 
better choices in the future. 

• I will deal with inappropriate behavior in ways that will not demean or humiliate. 

• Equal is not always fair. Consequences will be designed to fit the situation, and they 
may be different even when problems appear to be the same. 

Once developed, the goals are usually posted in the classroom and communicated to the students 
and their parents in a course syllabus. It is also possible for the teacher to engage the students in 
the development of the classroom rules, which may increase student understanding and sense of 
ownership. 

In addition to the general expectations outlined in class rules, CTE teachers may also develop 
procedures for beginning and ending the class period. The first few minutes of the class typically 
set the tone for the whole class period, so ensuring that students know what to do upon entering 
the class, how and when to take out equipment and materials, and what to do it they are late to 
class minimizes disruptions and gets students focused on the learning activities right away. 
Secondary students may also need guidelines for how to ask permission to leave the room. 
Because most CTE teachers have classrooms with equipment and lab areas, guidelines for use of 
the equipment may also be part of the classroom management plan. 

First Weeks of School 

Teachers must now implement these proactive measures and maintain them throughout the year. 
The following are guidelines for establishing a positive learning environment and preventing 
classroom problems, beginning with the very first day of class. 
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On the first day of class: 

• Greet the students as they enter the classroom or lab with a smile, a handshake, and a 
welcoming statement to your program or course. 

• Introduce yourself, perhaps with a bit of information about your occupational experience, 
licenses and certifications, colleges attended, etc. 

• Assign students temporarily to a seat. Usually alphabetic arrangement works fine for the 
beginning days of class. 

• Hand out a syllabus. Review the goals and objectives of the course. Highlight for the 
students major content, projects, and outcomes they can envision by completing the 
course requirements and any summative assessments that are required or offered. What 
can they expect to leam and what skills will they be able to do as a result of participating 
in this course? 

• Point out the “silent” curriculum procedures that are posted in the classroom and/or on a 
PowerPoint. 

• Assign the students to write a 3-5 page autobiography, asking for information such as a 
personal history, contact information (i.e., e-mail), any special accommodations needed, 
course schedule, likes and dislikes as related to school, school and community activities, 
hobbies and interests, reasons for taking this course, part-time and summer employment, 
plans for college and career, and any other information they would like the teacher to 
know. This may also take the form of a survey instead of a writing assignment. 

• Circulate as the students begin their writing assignment. 

Continuing on during the first and second week of classes: 

• Always, always start class on time. Be prepared. Have an opening activity that students 
are to do immediately upon entering the classroom. 

• Quickly learn and use students’ names. 

• Establish instructional set each day with a brief introduction of the objectives and 
activities for the day. Discuss expectations and what the student is expected to know and 
do as a result of the day’s instruction and activities. 

• Review during the first few days the plans for the course, expectations, envisioned 
outcomes, and the silent curriculum. Solicit input from students, especially about the 
silent curriculum. 

• Explain assessments that will be given, their purposes, and your role in helping them to 
achieve success. 
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• Discuss, particularly, workplace expectations for students enrolled in this unique program 
of study and who are preparing to be employed in related occupations. Expectations 
should include academic, technical, and affective competencies as well as the way in 
which occupations or jobs are organized in typical workplaces (e.g., through teams, 
independent problem solving, customer service, use of tools and equipment, the “rules” 
of employment, etc.). 

• Discuss any unique elements or components of the CTE curriculum, such as co-curricular 
student organizations, equipment safety, computer security, and use of power and tools. 

• Implement any routine that you wish to have followed throughout the year, such as 
handing out supplies and materials, getting the materials and equipment ready for the 
day’s work, rearranging classroom equipment or seats, clean-up, etc. It is good to discuss 
with students more appropriate or convenient ways to rearrange the classroom or lab. 

• Clearly communicate and model classroom rules or procedures. 

• Start immediately with having students complete assignments, write in a journal, begin 
projects, solve a problem, operate equipment, use tools, or other activities as appropriate 
to the curriculum and its objectives. Observe students as they work and practice in your 
classroom. Provide attention, help, and reinforcement as needed. 

• Hold students accountable, right from the beginning, for assignments, participation, 
cooperation, following routines, and acceptable behavior. Take action immediately if 
their actions are not acceptable by phoning parents, expecting all work to be turned in, 
and asking students to re-do work that is not to quality. 

• Draw on the autobiographies the students wrote during the first day or two of class and 
use the information appropriately. Never violate confidentially. But do point out how 
content and classroom activities will help students achieve their goals. 

• If possible, hold a brief conference with or interview each student to discuss important or 
interesting items in his or her autobiography. 

• Communicate with parents or guardians, via e-mail, phone, or letter. Attach the syllabus 
to written correspondence. Establish a web site and e-mail for communication with 
students, parents, and colleagues. 

• Provide positive reinforcement where warranted from the start and re-teach when it 
appears the student(s) is not learning the academic, technical, and/or silent curriculum. 
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Dealing with Inappropriate Behavior 

In spite of the best planning to prevent disruptions and misbehavior, beginning teachers need to 
be prepared to deal with situations in which students are acting inappropriately. There are four 
categories of disciplinary behaviors (Stage & Quiroz, 1997): 

1. Reinforcement 

2. Punishment 

3. No immediate consequences 

4. Combined punishment and reinforcement 

According to Marzano et al., (2003) reinforcement involves recognition and reward for positive 
behavior or for the timely cessation of negative behavior. Practices classified as punishment 
involve some type of negative consequences for inappropriate behavior, such as isolated time 
during class or after school or restriction of privileges. Interventions that are classified as “no 
immediate consequences,” as the name implies, involve some type of reminder when an 
inappropriate behavior appears imminent. For example, a student who typically engages in 
rowdy “horseplay” in a welding lab should be reminded before going into the lab what the 
appropriate behavior should be. Finally, the combined punishment and reinforcement involves 
recognition or reward for appropriate behavior in conjunction with punishment for inappropriate 
behavior. Teachers should strive for a healthy balance of these interventions so that none are 
used exclusively for all students. 

Often, a school has adopted an approach or policy on discipline that all teachers must use. 
However, it is usually up to the teacher’s judgment as to when each intervention is to be used. 
Two of these interventions most frequently used in schools are Think Time and Assertive 
Discipline. 

Think Time (Nelson & Carr, 1999, as cited in Marzano & Pickering, 2003) is a highly structured 
program with three basic goals: 

1. To provide consistent consequences across all teachers in the school when students 
engage in disruptive behavior 

2. To provide students with feedback for their disruptive behavior and to allow for 
planning to avoid future incidents 

3. To enable teachers and students to cut off negative social exchanges and initiate 
positive ones. (p. 33) 

Basically, in a Think Time classroom, students are given a quiet, isolated place to analyze their 
behavior and can return to the full classroom environment when they demonstrate they are aware 
of the behavior that led to the assignment of isolation and understand the appropriate alternative 
behaviors expected in the classroom and are willing to adopt those behaviors. 

Assertive Discipline (Canter & Canter, 1992, as cited in Marzano & Pickering, 2003) is widely 
used and based on traditional behavior modification approaches in which misbehavior results in 
specific consequences. The approach makes a sharp distinction between rules and directions. 
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Rules are in effect all the time whereas directions vary from activity to activity. Students’ off- 
task behavior is distinguished between disruptive and non-disruptive. If a student is off-task but 
not disrupting anyone, the teacher redirects the behavior without consequence. The approach also 
emphasizes appropriate behavior with negative consequences kept at a minimum. 

Assertive Discipline involves five steps. 

1. Establish a positive climate for discipline 

2. Practice assertive behavior 

3. Establish clear limits and consequences 

4. Follow through on consequences 

5. Implement a system of rewards or positive consequences for positive behavior 
(Marzano & Pickering, 2003, p. 34) 

As pointed out in the section of this paper on rules and procedures, the more clear and succinct 
the established classroom rules, the easier it is to communicate those expectations to students. 
Helping students be responsible for their own behavior through consistency and following 
through with discipline is part of the role of the CTE teacher in helping students attain the skills, 
both technical and interpersonal, to be successful in their future careers. The more opportunities 
that can be taken to communicate expectations for behavior to parents and students and 
consistently apply the classroom rules and consequences, the easier it will be to maintain 
classroom control and utilize positive interventions more often. 

Disciplinary Interventions and Consequences 

One of the most successful strategies a beginning teacher can master in classroom management, 
is the ability to minimize and defuse disruptions and inappropriate behavior when they are first 
observed and before they become real problems (Lieber, 2009). This involves the ability to 
redirect unproductive group talk with actions such as moving into close proximity of students 
who are having a sidebar conversation, knowing how to quickly acknowledge an occasional 
juvenile remark and move on without breaking the flow of the lesson, and prompting students 
with reminder questions to listen to others or to tone down hostile or argumentative statements. 
Lieber reminds beginning teachers that “adolescents will often go out of their way to say and do 
things that are provocative just to see how (teachers) will react and what (they) will do” (2009, p. 
195). The ability to defuse that behavior before it escalates into a major class disruption involves 
considering the student, the choices in the nature and timing of the teachers’ response, and 
whether the response will occur in front of the class or in a private moment later in the class 
period. Teachers can develop their own tool bag of responses that successfully maintain the 
delicate balance between continuing the class without interruption and stopping to correct 
inappropriate behavior. These tools can include everything from a brief, but directed look at the 
student, a quick acknowledgement of “I hadn’t thought of it that way,” or a brief reference to the 
life skill that needs to be developed and an invitation to self-correct. 

Beginning CTE teachers need to be prepared for problem behaviors which will require a higher 
level of intervention. Making a list of these behaviors is a good starting point (Lieber, 2009). 
They may include noncooperation and nonparticipation, student-to-student aggression, impulsive 
outbursts, and student-to-teacher aggression. These incidents can result in angry confrontations 
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for the student and for the teacher if left unchecked. The teacher’s role is to defuse and de- 
escalate these confrontations and power struggles by using as few words as possible to calm the 
situation, name the consequence of the behavior, and make a quick end to it. Trying to talk with 
the student immediately in front of the class often only escalates the situation. 

One-on-one conferences with students are an essential part of intervention. Time for these 
conferences can be at the end of class when other students are working on a final reflection 
activity, during lunch or prep periods, or before- or after-school sessions. Phone calls and student 
conference times are also possible. Conferences begin with describing the behavior that occurred 
and asking an open-ended question to prompt the student’s perspective. Focusing the student on 
replacing negative behavior with the desired behaviors brings the conference to it’s desired end: 
a behavior plan, learning contract, or checklist of desired behaviors that can be used in the 
upcoming days or weeks of class to support successful change in behavior (Lieber, 2009). 
Accountable consequences , different from the concept of punishment, are specific actions 
chosen to help students take responsibility for their behavior by correcting their mistakes or 
repairing the damage they caused (Lieber, 2009). These consequences promote shared 
responsibility: the student acknowledging and working on the development of positive behavior 
and the teacher supporting the student in making that behavior change. Beginning teachers need 
to develop at least three levels of consequences for supportive intervention: 

• Tier 1 involves immediate teacher responses to interruptions and redirecting minor 
problem behaviors. 

• Tier 2 addresses chronic unwanted behaviors and problem behaviors that can’t be 
resolved quickly. These consequences include conferences, mandated work sessions 
for individual practice or completion of work, owed time, time-outs, referral to the 
dean, case conferencing with other staff such as counselors, or contracts with daily 
checklists and feedback. 

• Tier 3 consequences involve automatic schoolwide responses that directly involve the 
principal, dean, or counselor. (Lieber, 2009, p. 13) 

Lab Management 

Bonfidini (1993) indicated that teacher preparation is a key element in eliminating student 
discipline problems. Yet managing students in a laboratory setting is more challenging in many 
ways than managing students in a traditional classroom setting. Preparation problems for the 
CTE teacher are compounded because instruction takes place in a complex learning environment 
which includes classroom, laboratory and often worksite learning. Teachers who fail to prepare 
classroom instruction properly have increased discipline problems. Administrators confirm that 
improperly maintained laboratories reduce the instructor's effectiveness and quality of student 
learning. Individual classroom management is the responsibility of the teacher; failure to 
properly prepare the physical environment and instructional time will decrease the probability of 
student problems. 

The laboratory setting creates unique challenges in creating rules and safety procedures than may 
not fit within the general disciplinary or dress codes of the school. Numerous legal cases have 
appeared in courts associated with dress and appearance codes. The courts again reaffirm the 
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need for reasonable rules relating to student dress. Failure to follow specific dress or grooming 
requirements has been upheld by courts when clear and necessary reasons have existed for 
establishing the regulation. Requiring students to wear or not wear specific clothing for safety 
reasons can be enforced. Many states have eye safety laws that require students to wear proper 
protective equipment. Students can logically be expected to wear hairnets, helmets, and other 
health and safety equipment in certain CTE courses and related student activities. Loose or 
baggy clothing, unbound long hair, or earrings could pose potential safety concerns in 
classrooms with moving mechanical parts, animals, or machinery such as saws, welding 
machines, or manufacturing equipment. The CTE teacher must consider these potential safety 
concerns and communicate them clearly in course syllabi and discipline plans. 

Examination of the yearly instructional plan is the first step in developing classroom 
management that includes laboratory management. Tailor the instructional plan to include the 
following elements: 

1. Assigned time sequence for student activities 

2. Identification of specific learning units to be taught, including safety and equipment 
use 

3. Listing of desired student performance objectives 

4. Instructional strategies for guiding the student through the learning process 
connecting classroom and laboratory learning experiences 

5. Listing of necessary classroom and laboratory materials, equipment, and supplies 

6. Procedures for laboratory maintenance 

7. Student and program evaluation 

Properly developing each of these items will eliminate many of the potential catalysts that create 
student discipline problems. The first item on the list is planning for the efficient use of 
instructional time. Realizing that time is a factor in establishing student and personal objectives 
can lead to better classroom organization, and consequently, less off-task and disruptive 
behavior. When laboratory time is well managed, it sends the message that that time is critical to 
developing the skills required for college and career. The process of selecting activities for CTE 
students can be an important factor in reducing idle time. The teacher should work through an 
activity before assigning it to a class. This should help determine the time factor required by the 
activity, taking into account the grade and ability of students. 

Having all necessary support elements available and ready to go, such as (a) plans, (b) 
instruction sheets, (c) references, (d) safety instruction, (e) audio visuals, and (f) materials and 
supplies, will enhance student success and further help to reduce discipline problems. In a 
laboratory classroom, the need for materials and supplies cannot be over emphasized. Failure to 
obtain sufficient planning and timely ordering and delivery of student materials available will 
create classroom problems. In many school systems, the teacher may face the problem of 
insufficient funds and lack of needed materials. Funding challenges require great efficiency in 
utilizing materials that are available. The teacher should conduct a financial audit of the yearly 
educational plan to determine if student activities can be accomplished within the financial 
framework. If the answer is negative, adjustments can be made to the plan or the teacher could 
seek additional help from advisory committees or local business partners. 
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Laboratory maintenance is the teacher's responsibility; that responsibility should be accepted 
without expecting others to control the environment. Although many support systems are 
available to assist in maintenance, the teacher who establishes his or her own maintenance 
control system and involves the students in implementing that system will reduce management 
problems in the laboratory. 

Providing Support for Students to Succeed 
Communication with Parents 

According to McEwan (2000), the more teachers inquire into and discover the particular and 
special qualities of their students, the easier it is to sustain atmospheres of trust and respect 
within their classrooms and with parents. The key to effective communication with parents is 
taking a real interest in the student and communicating that caring and concern to parents. 

Maintaining contact with parents takes time and effort, but it is well worth it. Understanding the 
role that parents play in supporting their children’s efforts goes a long way to ensuring the 
students are successful. Remembering the following can make it easier to work with parents: 

• All families have strengths 

• Parents can provide important perspectives about their student 

• Parents as partners with a teacher can result in greater support for students 

Teachers and parents have shared goals. Both are committed to the development and education 
of children. To communicate that shared sense of purpose, teachers can show respect for parents 
in tone of voice, expressions, and body language and express the intent that they wish to be a 
partner in the education of the child. The impressions a parent has of the teacher will dictate the 
parent’s willingness to support the expectations of the learning environment and any disciplinary 
interventions that may be required. 

CTE teachers have unique opportunities to form early relationship with parents. As suggested 
earlier in this paper, writing a letter of introduction can be a great way for parents to understand 
the teacher’s intent to make them partners in their students’ success. Inviting parents to an open 
house in the classroom and providing an orientation to the program is a great way to set 
expectations and also take advantage of natural curiosity because CTE is a very different 
learning environment from an academic classroom. Inviting parents to see work done by their 
students or having students take or email photos of projects are effective ways to involve parents 
in celebrating their children’s progress. 

To develop a working relationship with parents, CTE teachers can engage parents in reviewing 
students’ work as often as possible. Parent nights, on which parents assist children with the CTE 
laboratory work or demonstrate finished work, are an effective means of informing parents of the 
kinds of knowledge and skills students are learning, as well as the conduct required of students to 
be successful in further learning and the workplace. 
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Teachers will be successful with parents when they communicate good news more often than 
bad. Notes and phone calls about how students are meeting learning goals are welcome 
communication to parents. The CTE teacher has a unique opportunity to know students better 
than other teachers in the school because the CTE course deals with students’ career aspirations 
and dreams and the teacher of that course is in a mentoring role. This is a key relationship that 
should be communicated with parents and forms a supportive environment not only for the 
student, but also for the teacher. 

Extra Help 

As suggested earlier in this paper, believing that all students can learn is a fundamental 
understanding for successful teaching. This belief drives teachers to set high expectations for all 
students. But setting high expectations is not, in and of itself, sufficient for all students to achieve 
those expectations. Some students will need more time and support than others. Consequently, 
effective teachers plan to provide extra help to students who do not meet those expectations on 
the first try. Several concepts are fundamental to providing effective extra help. 

• All students are capable of quality work. 

• Extra effort may be required to meet standards. 

• Effective extra help motivates students to do their best. 

In order to be effective, extra help must be offered early, frequently, and regularly. It must be 
easy to access. The system of extra help must be closely planned and monitored, or it will not be 
effective in helping students reach their goals. CTE teachers should be prepared with procedures 
for immediately working with students when work is not completed or does not meet standards. 
One of the most effective strategies is to require students to re-do that work, correcting mistakes 
and clarifying misconceptions so that the skill or concept is mastered. Re-doing work is more 
than simply repeating an assignment. Teachers need to plan for re-teaching opportunities, either 
in the classroom while other students are working on different learning activities or in extra help 
or after-school time periods if they are available. Students need help in managing re-do 
opportunities through clear timelines for the re-doing of the work, progress and feedback reports, 
and celebration when the work reaches standards. 

CTE teachers need to develop specific procedures they will use to intervene when a student’s 
grade falls below the proficient level, such as making a phone call home or sending an alert form 
to parents to inform them about the extra help prescribed and what the student needs to do to 
bring up the grade. Involving students in keeping track of their own progress toward meeting 
standards will help them be aware of their need for extra help and be proactive in seeking it. If 
poor work persists, the CTE teacher may seek the help of the student’s guidance counselor to see 
if poor performance is a problem in other courses. If so, a conference with the student, his or her 
parents, and the counselor can be used to develop a success contract for improved performance. 

Career and Technical Student Organizations 

Career and technical student organizations (CTSOs) play a major role in establishing a classroom 
environment in which the students are partners. CTSOs prepare young people to become 
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productive citizens and to assume roles of leadership in their communities by practicing those 
leadership roles in the CTE classroom. These organizations provide a unique program of career 
and leadership development, motivation, and recognition for secondary and postsecondary 
students enrolled, or previously enrolled, in CTE programs. 

Educators have found that the CTSO is a powerful instructional tool that works best when it is 
integrated into the CTE curriculum. The CTE instructor provides organized curriculum-oriented 
activities that help students gain career, leadership, and personal skills that maximize 
employability and the ability to become productive citizens in the workforce, home, and 
community. Preparing for contests requires extra effort and often extra help from peers and 
teachers in learning content to a deeper level than students may have first attained. Using the 
motivation of competition along with extra support and encouragement can have an effect on 
student learning. CTSOs themselves provide excellent resources for teachers to accomplish the 
integration of the organizational structure and learning activities that successfully drive 
implementation. 

CTE Teachers as Mentors in an Advisory Program 

Although much of creating a well-managed classroom is about the student-teacher relationship 
and particularly in a CTE classroom, about the teacher serving as a career mentor, the CTE 
teacher may work in a school in which there is a formal program for teachers serving as advisors 
or mentors to students. In Things That Matter Most in Improving Student Learning (Bottoms, 
1998), a teachers as advisors program is identified as one of the strategies school use to make a 
difference in student achievement. Many teachers believe that advising students about enrolling 
in academic and CTE courses in high school is a school counselor’s responsibility. The fact is 
that most high schools do not have enough counselors to provide students and parents with 
ongoing information and advisement about education and career opportunities. The CTE teacher 
is in a unique position of having direct knowledge from industry and from their own experience 
to (a) advise students and parents about the need to take more difficult academic and CTE 
courses, (b) relate the courses to various careers and postsecondary studies, and (c) offer ways 
that parents can help students reach their goals. 

Research indicates (Finn, 1989, 1993) that students who are not connected to an adult, who do 
not have a goal beyond high school and who are not in some extracurricular activity are less 
likely to finish high school. Every student deserves to have a caring adult who advocates for 
them and guides them into adulthood, to help every student form goals with parent involvement 
and connect students to some activity and group beyond the classroom. Some of the steps a 
teacher-advisor can take to support students are: 

• Call parents when students are absent 

• Involve parents in supporting their children’s education 

• Ensure students’ work is meeting course standards 

• Connect students to extra help 

• Advise students on academic and CTE coursework for success in college and career 

• Collaborate with school guidance and counseling staff for support and options for 
referral for problems 
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There is an emerging consensus in the school reform literature about the conditions that 
contribute to student success (Annenberg Institute for School Reform, 1994; Darling-Hammond, 
1994; as cited in Klem & Connell, 2004). These conditions include: 

• High standards for academic learning and conduct 

• Meaningful and engaging pedagogy and curriculum 

• Personalized learning environments 

For students to take advantage of higher expectations and more advanced curricula, they need 
support from the people with whom they interact in school (Lee, Smith, Perry, & Smylie, 1999; 
both as cited in Klem & Connell, 2004). Teachers and schools that provide these supports are 
more likely to have students who are engaged in and connected to school. 
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Appendix 2E 



Flowcharts of Module Units and Lessons 



Units and Lessons in the Instructional Planning Module 




Units and Lessons in the Instructional Strategies Module 
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Units and Lessons in the Classroom Assessment Module 
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Units and Lessons in the Classroom Management Module 
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Appendix 2F 

Teacher Self-Efficacy Scale Results 



Table 2F.1 

Pre-Post Teacher Self-Efficacy Scale (TSES) Sub-Scale Analysis by Paired Samples t-Test 



TSES Sub Scale 


Pre 


Post 


Change 


Statistics 
P t 


Efficacy in Student Engagement 


6.40 


6.86 


0.46 


.030 


2.35 


Efficacy in Instructional Strategies 


6.49 


7.13 


0.64 


.004 


3.33 


Efficacy in Classroom Management 


6.60 


7.23 


0.63 


.004 


3.29 



Tables 2F.2 

Item-by-Item Pre-Post Responses on TSES Sub-Scales Analysis by Paired Samples t-Test 



Efficacy in Student Engagement Sub-Scale Item 


Pre 


Post 


Change 


P 


How much can you do to get through to the most difficult 
students? 


6.63 


6.84 


0.211 




How much can you do to help your students think 
critically? 


7.00 


7.16 


0.158 




How much can you do to motivate students who show 
low interest in school work? 


6.00 


6.42 


0.421 




How much can you do to get students to believe they can 
do well in school work? 


6.89 


7.16 


0.263 




How much can you do to help your students value 
learning? 


6.47 


6.95 


0.474 




How much can you do to foster student creativity? 


6.16 


7.21 


1.053 


0.003 


How much can you do to improve the understanding of a 
student who is failing? 


6.21 


6.68 


0.474 




How much can you assist families in helping their 
students do well in school? 


5.84 


6.47 


0.632 




Efficacy in Instructional Strategies Sub-Scale Item 


Pre 


Post 


Change 


P 


How well can you respond to difficult questions from your 
students? 


7.16 


7.37 


0.211 




How much can you gauge student comprehension of what 
you taught? 


6.37 


7.21 


0.842 


0.019 


To what extent can you craft good questions for your 
students? 


6.37 


6.89 


0.526 




How much can you do to adjust your lessons to the proper 
level for individual students? 


6.26 


6.89 


0.632 




How much can you use a variety of assessment strategies? 
To what extent can you provide an alternative explanation 


5.89 


7.00 


1.105 


0.002 


or example when students are confused? 


6.95 


7.53 


0.579 




How well can you implement alternative strategies in your 


6.26 


6.79 


0.526 
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classroom? 


How well can you provide appropriate challenges for very 
capable students? 


6.63 


7.37 


0.737 




Efficacy in Classroom Management Sub-Scale Item 


Pre 


Post 


Change 


P 


How much can you do to control disruptive behavior in the 
classroom? 


6.63 


7.00 


0.368 




To what extent can you make your expectations clear about 
student behavior? 


7.11 


7.84 


0.737 


0.018 


How well can you establish routines to keep activities 
running smoothly? 


7.11 


7.21 


0.105 




How much can you do to get students to follow classroom 
rules? 


6.58 


7.42 


0.842 


0.003 


How much can you do to calm a student who is disruptive 
or noisy? 


6.32 


6.95 


0.632 




How well can you establish a classroom management 
system with each group of students? 


6.32 


7.11 


0.789 


0.001 


How well can you keep a few problems students from 
ruining an entire lesson? 


6.37 


7.05 


0.684 




How well can you respond to defiant students? 


6.37 


7.26 


0.895 


0.003 
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Appendix 2G 

Focus Group and Instructor Debrief Protocols 

Focus Group Protocol 1 

1. What is the first word that comes to mind when someone asks you how your first year as 
a CTE teacher went? 

2. Think back over this school year. What day or event are you most proud or fond of? 

3. Talk about the kinds of training - both topic and format - that you would design if you 
were to prepare new CTE teachers, like yourself, for the classroom. 

4. We know that success in the classroom is influenced by many factors. Take a look at this 
list we’ve just distributed. Which of these would you say is the factor that most 
influences your success in the classroom? [Raise hands] 

a) Having the right knowledge and training 

b) Having a supportive principal or instructional leader 

c) Having a colleague who acts as a mentor 

d) Having other new CTE teachers with whom to share stories 

e) Something else 

5. What aspects of [factor] do you feel contribute to your success? 

6. Imagine you’ve been invited to a meeting of the State Administrators Association 
attended by superintendents, principals, and tech center directors. You are giving a 
breakout session for schools and tech centers hiring new CTE teachers this fall. What will 
you tell them they should do to help those teachers be successful with students? 

7. What should new CTE teachers be expected to be able to do in terms of instructional 
planning? Here is a list of new teacher competencies from INTASC. We will go through 
them one by one and I will ask you to rate each one as necessary and reasonable (Green 
card); necessary but unreasonable (Yellow card); and unnecessary and unreasonable (Red 
card). 

8. What kinds of support do you think would help to make the yellow card items more 
reasonable? 

9. Regarding the workshop you attended today: what aspect or topic discussed today do you 
most wish you had at the beginning of the school year? 

10. Finally, of all the things we’ve discussed tonight, what do you feel is most important for 
the developers of CTE teacher training? 
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Focus Group Protocol 2 



1. What is the first word that comes to mind when someone asks you how your first year as 
a CTE teacher went? 

2. Some of you may have heard of Twitter, a social networking phenomenon that involves 
people posting tidbits like a status update in 140 characters or less. What would you tweet 
about the training you’ve been attending? In other words, if you only have 5-8 words, 
what would you say? 

3. Let’s talk about some specific characteristics of this training. Please use your red, yellow, 
or green cards to indicate whether you thought each aspect was right on (green); just OK 
(yellow); or needs major revision (red): 

a) Vocabulary used by the instructor or in materials 

b) Clarity of the message communicated by instructor or in materials 

c) Sequence of topics 

d) Pace of instruction 

e) Quality of examples given 

f) Differentiation/accommodation of your individual needs 

4. Tell me about a time during the workshop today when you didn’t understand something 
in the materials or that the instructor was saying. 

5. How do you think you will use the materials from this training when you return to your 
classroom? 

6. If you think you won’t use these materials or the content you learned in the training, tell 
me why. 

7. What aspects of this training could you do without? 

8. What do you wish was included in this training that wasn’t covered? 

9. Did you feel you had enough time to explore the concepts in this training? 

10. Think back to the time during the training that you were working on [an artifact]. What 
do you feel you learned through that process? What additional support will you need to 
use that artifact in your classroom? 

11. Describe the process of developing [the artifact]. Was it helpful? Was it frustrating? 

12. Finally, of all the things we’ve discussed tonight, what do you feel is most important for 
the developers of CTE teacher training? 
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Daily Instructor Debrief Protocol 

1. How closely would you say today’s training met with your expectations and plans? 

1 (Not at all) 2 3 4 5 (Exactly) 

2. What did you do that deviated from what you planned? 

a. What did you see, hear or learn to determine that a change was needed? 

b. How did you change the planned activity? 

c. How did the change work out? 

d. What would you do differently next time? 

e. What materials can/should be developed to support this adaptation? 

f. Is this adaptation likely to be something a future instructor will need to make? 

g. Should the module be changed, or should this adaptation be added as an option? 

3. What would you like to change about the module materials based on your experience 
today? 

4. What aspects of today were most successful? Why do you think so? 
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Appendix 3A 



Results of Year 1 Five-State Survey, Summary of Survey Responses 5 

Section I: Use of Assessments 



1. Do your CTE students take an end-of-program technical assessment (that specifically 
measures his/her occupational/technical skills and knowledge)? 





Administrators 


Teachers 


Yes 


89.6% 


92.3% 


No 


10.4% 


7.7% 



2. Which of the following best describes the assessment they take? (Check al 


1 that apply 




Administrators 


Teachers 


Teacher-designed classroom assessment 


37.5% 


25.6% 


State-developed standardized assessment 


31.3% 


30.8% 


National standards-based assessment 


62.5% 


51.3% 


Certification test or licensure exam for business or 
industry 


62.5% 


38.5% 


Other 


4.2% 


7.7% 



3. How well do you feel the end-of-program assessment matches the content of your 
program(s)? (Teachers, check one. Administrators, check all that apply.) 





Teachers 


Administrators 


Very well 


28.2% 


18.8% 


Satisfactory 


59.0% 


52.1% 


Not very well 


5.2% 


25.0% 


Not at all 


0.0% 


2.1% 


Not sure 


0.0% 


6.8% 



4. Aside from formative assessment, which one of the following best describes the testing 
process for your program(s)? 





Administrators 


Teachers 


Pretest and posttest 


20.8% 


48.7% 


Posttest only 


25.0% 


28.2% 


Some programs use a pretest and posttest, others posttest only. 


43.8% 


15.4% 



5. For what purposes does your school use technical assessment data (check all that apply)? 



Administrators 



Teachers 



5 In cases where survey percentages add up to less than 100%, this is due to some respondents electing not to answer 
the question. 
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To help determine whether or not a student graduates 


12.5% 


25.6% 


To recognize high-achieving students 


41.7% 


41.0% 


To maintain a continuous improvement process 


77.1% 


71.8% 


To include in the school's overall process for accreditation 


45.8% 


43.6% 


To include as a part of teacher evaluations 


20.8% 


17.9% 


To help document school and program progress 


66.7% 


59.0% 


To make improvements to programs in areas in which scores 
are weak 


70.8% 


64.1% 


To make improvements to instruction of individual students 
who are weak 


41.7% 


43.6% 


To report to outside bodies such as the District, State 
Department of Education or the Federal government 


75.0% 


64.1% 


To report to reform organizations, such as High Schools That 
Work 


14.6% 


17.9% 


To give credit for program completion 


31.3% 


41.0% 


To help students receive certification for the job market 


68.8% 


64.1% 


To determine student grades 


27.1% 


41.0% 


Other 


2.1% 


0.0% 


Not sure 


2.1% 


5.1% 



6. Does your school use technical assessments that include integrated academics scores in 
addition to technical scores? 





Administrators 


Teachers 


Yes 


22.9% 


35.9% 


No 


50.0% 


10.3% 


Not Sure 


16.7% 


46.2% 



7. How are the academic data used? (Check all that apply) 





Administrators 


Teachers 


To help determine whether or not a student graduates. 


6.3% 


12.8% 


To recognize high achieving students (e.g., special designation 
on diploma or some other form of recognition). 


8.3% 


15.4% 


To maintain a continuous improvement process. 


20.8% 


30.8% 


To include in the overall process for accreditation by the regional 
agency. 


6.3% 


10.3% 


To include as a part of teacher evaluations. 


2.1% 


2.6% 


To help document school and program progress. 


18.8% 


20.5% 


To make improvements to programs. 


14.6% 


23.1% 


To make improvements to instruction of individual students. 


14.6% 


20.5% 


Data are reported to outside bodies such as the District, State 
Department of Education, or the Federal government. 


8.3% 


10.3% 


To report to reform organizations, such as High Schools That 
Work 


4.2% 


5.1% 
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To give credit for program completion. 


10.4% 


10.3% 


To determine student grades 


8.3% 


23.1% 


Not sure 


0.0% 


0.0% 


Other 


0.0% 


0.0% 



8. Who either has the assignment or volunteers to facilitate data-based decision making within 
your school? (Check all that apply.) 





Administrators 


Teachers 


A teacher 


47.9% 


20.5% 


An assessment coordinator 


14.6% 


17.9% 


An IT coordinator 


2.1% 


7.7% 


A curriculum coordinator 


18.8% 


5.1% 


A data analyst 


4.2% 


0.0% 


An administrator 


77.1% 


61.5% 


Other 


12.5% 


5.1% 


No one has the assignment or volunteers to do 
this 


4.2% 


7.7% 



9. What services do the coordinator(s) or facilitator(s) provide to help support the use of 
technical and/or academic assessment data by administrators and teachers? (Check all that 
apply-) 





Administrators 


Teachers 


Collects and shares information about the test content 


58.3% 


30.8% 


Matches content of tests with standards 


22.9% 


2.6% 


Matches content of tests with curriculum 


33.3% 


5.1% 


Collects the data into one form 


39.6% 


25.6% 


Summarizes the data at a classroom or program level 


56.3% 


41.0% 


Summarizes the data at a school level 


54.2% 


35.9% 


Communicates the results to teachers 


66.7% 


59.0% 


Assists teachers and/or administrators in interpreting the results 
of the data 


54.2% 


48.7% 


Other 


2.1% 


5.1% 



Section II: Use of Assessment Data 

10. Which statement most accurately represents the amount of student assessment data you 
receive? 





Administrators 


Teachers 


Do not receive assessment data 


4.2% 


7.7% 


Too little to be useful or informative 


14.6% 


15.4% 


The appropriate amount 


66.7% 


66.7% 


Too much to deal with or use effectively 


4.2% 


7.7% 


Other 


10.4% 


2.6% 
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11. Who provides this data to you? 





Administrators 


Teachers 


An administrator 


14.6% 


25.6% 


Teacher(s) 


4.2% 


0.0% 


Another school person (e.g. data analyst, curriculum coordinator, 
IT coordinator, assessment coordinator) 


29.2% 


46.2% 


The assessment vendor company 


29.2% 


10.3% 


A district representative 


0.0% 


0.0% 


A state department representative 


4.2% 


0.0% 


A consultant 


0.0% 


0.0% 


I provide it for myself 


8.3% 


5.1% 


Other 


6.3% 


5.1% 



12. When you receive a report of technical assessment data, does it have an analysis of results by 
objective /skill in addition to overall scores? 





Administrators 


Teachers 


Yes 


52.1% 


66.7% 


No, but I wish it did. 


39.6% 


20.5% 


No, and I don’t need 
that. 


4.2% 


5.1% 



13. When you receive a report of technical assessment data, are the data broken down(or easily 
broken down) by class and groups of students? 





Administrators 


Teachers 


Yes, they are broken down. 


50.0% 


69.2% 


No, but I can easily break them down by looking at the 
spreadsheets. 


18.8% 


10.3% 


No, but others who handle the data are willing to break them 
down by teacher or administrator request. 


12.5% 


5.1% 


No, they are only provided in a summary form and cannot be 
broken down. 


12.5% 


2.6% 



14. Do you do data analysis on your own, or with colleagues? 





Administrators 


Teachers 


I don’t do data analysis 


10.4% 


23.1% 


By myself 


27.1% 


48.7% 


With 2-6 colleagues 


47.9% 


17.9% 


With 7-10 colleagues 


2.1% 


2.6% 


With more than 10 
colleagues 


8.3% 


0.0% 



15. Who analyses data with you? (Check all that apply.) 
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Administrators 


Teachers 


Teachers in own program 


39.6% 


12.8% 


Teachers in other CTE 
programs 


22.9% 


5.1% 


Teachers in academic 
programs 


12.5% 


2.6% 


Administrators 


56.3% 


7.7% 


School assessment coordinator 


29.2% 


5.1% 


District assessment 
coordinator 


0.0% 


0.0% 


Data experts 


4.2% 


0.0% 


Assessment experts 


2.1% 


0.0% 


Test vendor representatives 


2.1% 


2.6% 


Other 


8.3% 


0.0% 



16. Do you use technical assessment data to make or request instructional decisions, either for 
individual students or for a class as a whole? 





Administrators 


Teachers 


No, but I would not be likely to use them 


6.3% 


5.1% 


No, but I believe I should be using them 


22.9% 


23.1% 


Yes 


68.8% 


69.2% 



17. How often during the school year? 





Administrators 


Teachers 


More often than twice 


33.3% 


17.9% 


Twice 


18.8% 


15.4% 


Once 


18.8 


38.5% 


Never 


0.0% 


0.0% 



18. Please explain the process you use to make or request instructional decisions 
Administrators 

• Look for the weak points and try to build those areas up in instruction. 

• We match local curriculum to NOCTI Assessment blueprints. 

• The processes are identified in a management processes manuals which provides a 
written procedure and process map. These processes are audited twice a year by a team of 
auditors. The auditors are professional staff members. An external audit is held once per 
year. For these audits corrective actions, continuous improvement, and corrective actions 
are developed. The administrator or staff member responsible for each process is required 
to respond and revise the process to be in compliance with the audit recommendations. A 
minor infraction, if not corrected, can move to a major infraction. Major infractions are 
dealt with by the Director. 

• I compare the data with my learning guides as a cross walk. I revise my learning guides 
accordingly. 
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• Teachers use practice assessments for industry certifications and from that data we 
determine students’ strength and weaknesses and make instructional changes. 

• Technical assessments given at the end of the year are autopsy reports. The scores cannot 
help students leaving the class, but can help with curriculum revisions for areas that were 
weak. Our internal academic pre testing allows for instructors to adapt curriculum and 
support for students who need it. We implement assessment for learning and these 
strategies help with changing the instruction as needed for individual students. End of 
program tests, like NOCTI are not helpful for current student instruction. 

• Administrators, Guidance Counselor, Special Populations Coordinator and Test 
Coordinators review and analyze data. The Assistant Director and Guidance Counselor 
meet individually with program instructors to analyze data further and make instructional 
decisions/ 

• Fall goal setting and spring program evaluations 

• Formal written requests. 

• Check data and talk with instructors. 

• Based on assessment data, requesting modified delivery of curriculum may occur— 
allowing more time for student(s), might even result in asking a teacher to improve their 
technical skills. 

• I use this information to place other students in the DATA program. 

• Collaboration with administrator, counselor, instructors 

• Teacher analyzes the data with administrator and counselor. Make proposal to the 
director. Decision would be made at the end of the school year before the start of the next 
school year with everyone involved. 

• Industry driven by the advisory board. 

• it is a cycle. Fook at data, make changes to instructional practices, implement changes, 
collect data, and then start process all over again. 

• I look more at the student placement and completion percentages than at individual 
assessment scores. 

• Team of CTE, academic, and special education teachers and administrators 

• We use short cycle assessments and review the data almost weekly 

Teachers 

• I look for common areas that students scored poorly and then I review my lessons for that 
theme. Monitor and adjust. 

• Match percentage of time spent in class to results on assessment and base my request on 
areas in which students scored the worst. 

• Revise curriculum to meet the instructional needs and objectives. 

• I review data from that school year's exam and make the appropriate adjustments in next 
years curriculum. 

• Data analysis and dialogue with administrator/ assessment coordinator 

• If students as a group, do poorly on a particular area I try the next year to increase the 
content instruction 

• Fook for areas of poor skills and suggest methods or needed equipment to improve 
scores. 
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• Review the student scores.; 2. Determine what areas students are struggling.; 3. Adjust 
instruction to meet student needs. 

• The Craft Advisory Committee helps the teacher make these decisions. 

• I use test results to determine weak areas. 

• Additional instruction is based on deficient areas on technical assessments 

• State 6G pipe welding Certification every May. The number of graduating students and 
how well they did make most decisions. 

• Student performance. 

• If there is a competency that I teach and I need materials or equipment to help teach the 
students better I discuss this with my administrator as well as my O.A.C committee. 

• Collaboration with other CTE teachers and curriculum specialist 

• We usually request a meeting with the administrator of our building and have a meeting 
regarding the material. It usually takes place a day or two after the request. 

• I adjust instructional methods used based upon success and the rate of success of 
individual students on technical assessments. 

• Program faculty with input from Advisory Board 

• I use test results to determine weak areas. 

• If students as a group, do poorly on a particular area I try the next year to increase the 
content instruction 

• Check to see where students need help and try to make changes for the next year. 

• I align my program with the NOCTI tasks that are tested during the students senior year. I 
am relatively free to make the decisions that I deem appropriate. I share my plans and 
request input from my OAC regarding relevance of curriculum and sequence of 

• Collaboration with other CTE teachers and curriculum specialist 



19. Who directs/leads you in this process? 





Administrators 


Teachers 


Teachers for their own CTE program 


16.7% 


35.9% 


A teacher leader 


0.0% 


0.0% 


An instructional leader 


2.1% 


0.0% 


The school principal 


8.3% 


7.1% 


The CTE director 


16.7% 


7.1% 


The school assessment coordinator 


4.2% 


3.6% 


The curriculum coordinator 


0.0% 


7.1% 


The school IT coordinator 


0.0% 


0.0% 


An advisory board member 


4.2% 


0.0% 


Other 


8.3% 


7.7% 


No one does this. 


10.4% 


10.3% 
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20. How did you learn to use technical assessment data to make instructional decisions? (Check 
all that apply.) 





Administrators 


Teachers 


During teacher or administrator training 


31.3% 


17.9% 


Professional development programs while a teacher 


10.4% 


15.4% 


Mentoring with someone skilled in using the data 


4.2% 


5.1% 


Self-taught 


18.8% 


30.8% 


I do not feel I know how to use technical assessment score 
data. 


0.0% 


2.6% 


Other (please explain) 


6.3% 


0.0% 



21. What changes have you made or requested in instruction of the class as a whole on the basis 
of analysis of the data (check all that apply) 





Administrators 


Teachers 


Requested that business advisory committee members help 
address problem areas 


35.4% 


28.2% 


Requested additional supplies or equipment 


39.6% 


35.8% 


Asked for additional support and ideas from other 
teachers/administrators 


45.8% 


25.6% 


Discussed curriculum relevance and alignment with standards and 
assessments with peers 


45.8% 


28.2% 


Discussed appropriateness of the assessment with other teachers 
in your area or with administrators 


45.8% 


25.6% 


Re-evaluated textbooks and learning materials based on the 
results of assessment 


29.2% 


43.6% 


Added more projects and exercises in areas in which the group 
scored low 


45.8% 


48.7% 


Changed lesson plans to place more emphasis in areas in which 
the group scored low 


47.9% 


56.4% 


Other 


0.0% 


0.0% 



22. What are examples of changes you have made or requested in individualized instructional 
strategies for particular students as a result of the data analysis? (Check all that apply.) 





Administrators 


Teachers 


Provided students with additional assistance during class in 
area in which they performed poorly 


52.1% 


66.7% 


Provided poorly performing students with materials on test- 
taking skills and strategies 


39.6% 


35.9% 


Teamed up low-performing students with students who 
performed better in those areas 


25.0% 


38.5% 


Provided high-performing students with additional, more 
challenging projects and/or readings 


27.1% 


28.2% 


Emphasized students’ strengths to motivate them 


33.3% 


51.3% 
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Other 



10.4% 



5.1% 



23. If you have made or requested changes to your instruction or curriculum based on data, how 



effective have you generally found them to be? 





Administrators 


Teachers 


Very effective in improving student learning and test scores 


12.5% 


23.1% 


Somewhat effective in improving student learning and test scores 


43.8% 


33.3% 


Ineffective — student learning did not improve and neither did 
test scores 


0.0% 


0.0% 


Seemed to improve student learning, but test scores did not 
improve 


2.1% 


14.3% 


Improved test scores, but do not think it improved student 
learning 


0.0% 


0.0% 


I am unsure 


8.3% 


5.1% 



24. Do the administrators and teachers meet together to discuss using technical assessment data 
to make instructional decisions? 





Administrators 


Teachers 


Yes 


64.6% 


43.6% 


No 


6.3% 


28.2% 



25. How often? 





Administrators 


Teachers 


Twice a year or more 


47.9% 


23.1% 


Once a year 


14.6% 


17.9% 


Once every two-three years 


0.0% 


0.0% 


Less than once every three years 


0.0% 


2.6% 


Other 


2.1% 


0.0% 



26. About how many hours per year of released time are provided for teachers to work on data 
analysis or interpretation to make data-based judgments and decisions? 





Administrators 


Teachers 


10 or more hours 


14.6% 


5.1% 


7 to 9 hours 


8.3% 


2.6% 


4 to 6 hours 


4.2% 


10.3% 


1 to 3 hours 


12.5% 


17.9% 


None 


31.3% 


33.5% 
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27. About how many hours per year do you provide for yourself to work on data analysis or 



interpretation to ma 


ce data- based juc 


gments and 




Administrators 


Teachers 


10 or more hours 


47.9% 


33.3% 


7 to 9 hours 


8.3% 


2.6% 


4 to 6 hours 


4.2% 


12.8% 


1 to 3 hours 


8.3% 


20.5% 


None 


2.1% 


2.6% 



28. In addition to standardized technical assessment data, what other types of data do you 
collect/use to facilitate data-based decision making? 





Administrators 


Teachers 


Student academic scores 


10.4% 


7.7% 


Student performance in classroom activities (e.g., projects, 
quizzes, discussions) 


20.8% 


43.6% 


Transcript data, such as grades, grade point average, courses 
completed 


6.3% 


0.0% 


Program dropouts 


2.1% 


3.6% 


Factors such as community environment, CTSOs, student 
special needs, student background, attendance, behavior etc. 


22.9% 


17.9% 


Other 


8.3% 


7.1% 



29. If you do not use data-driven decision making, please explain why. 

Administrators 

• As an administrator, making "instructional" decisions usually falls on the teacher. I use 
data for other purposes. 

• We make decisions for our core classes based on assessment data, but we frequently do 
not have much assessment data from our vocational classes beyond what our local 
teachers provide. I know that our vocational coordinator had a few students take a 
standardized test this spring based on the classes they had been enrolled in, but that was 
the first of that kind of testing I've been aware of in 5 years as principal. 

• I am a Counselor. The Administration reserves these decisions not to include the 
Counselor. 

• We use in certain programs but not in all. 

• We are beginning to use it but need more professional development and data sources to 
become more proficient in it utilization. 

• It takes time to analyze data and I don't have the time needed to do it. 

• We DO use data-decision making but do not receive technical data (e.g. related to 
Woods, Welding, etc. Instead we use more academic area testing (reading, math, science, 
writing), some of which is related to the tech areas (e.g. Reading for Information) and 
some of which is not 

• We make decisions for our core classes based on assessment data, but we frequently do 
not have much assessment data from our vocational classes beyond what our local 
teachers provide. 
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• We are in the beginning stages of data-driven decision making. We established Teacher 
Data Teams in the 08-09 school year and have begun reviewing data from the STAR test 
and from TSAs. 

• Not enough time to analyze or collect information to analyze 

• Teacher, program and students have been performing at an acceptable level. Instructor 
discusses with me any concerns in individual student learning. 

• The technical exams provided by the state department are many times not up to date with 
material instructed. Assessment is too varied in source, nature and content to accumulate 

Teachers 

• Students learn what the industry needs as well as the community as well as individual 
goals of the student. 

• I just taught my first year in the school/program and collected the date. I will use it to 
better structure my curriculum for next year. 

• Based on the "Standardized Test" - 1 do not put any validity in it. I have not seen the test, 
they won't let us(teachers) see the test and based on some of the questions my students 
ask me upon returning, some of the questions are not relevant, up to date or are too in 
depth for students. 

• Most tests are not program- specific. 

• Currently, I do not use data driven decision making strategies, however I am planning to 
revise my program curriculum to address this issue. 

30. Do you intend to use data-driven decision making in the future? 





Administrators 


Teachers 


Yes 


81.3% 


71.8% 


No 


4.2% 


7.7% 


Not Sure 


12.5% 


17.9% 



31. How could standardized technical assessment data reports be improved to make them more 
useful for decision making? 

Administrators 

• First we need the test to measure what we need them to measure. 

• Additional breakdown by skill. 

• Larger Print; Breakdown data in other forms such as pie charts, Gantt, Paretto charts 

• Align cut off scores with national scores. Some NOCTI cut off scores are skewed. 2. 
Include business and industry in test design for relevancy. Some NOCTI emphasize a 
small segment of a course. 

• An individual report for each student should be available to show strengths and 
weaknesses in the different content areas covered by the assessment. 

• The loudest complaint I hear about NOCTI is that the tests questions are out of date. 
Teachers have no way of knowing this except what students report after testing. I don't 
know how to solve this issue. Having more detailed test blueprints might help teachers 
align competencies to the blueprint. 

• Not sure. 
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• If data reports provide a clear indication of specific expected learning outcomes, how our 
students have performed, and suggestions for improvement, then they will be useful. 

• Accessible by computer and able to pull reports on an as needed bases. 

• Have more teacher and curriculum input on the assessment 

• The reports themselves are OK. Providing additional training for teachers in 
interpretation of test scores would help. 

• Standardized technical assessment data needs to generate reports on an individual basis, 
to a program basis, to a school wide basis, to a state wide basis and finally on a national 
basis and possibly an international basis depending on the career area. We need to be able 
to see how each and every student that completes a program compares to every other 
individual in their cohort class. 

• Results should be tied to specific skill sets/standards. 

• It would increase the credibility of the tech areas as the other traditional ed areas already 
use data (e.g. English, Math, etc) 

• If data reports provide a clear indication of specific expected learning outcomes, how our 
students have performed, and suggestions for improvement, then they will be useful. 

• make available online 

• At the program level, it would help to match the assessment with the curriculum. 

• Our current assessments do not give adequate information that can be analyzed 

• Correlate the data to effective practices that impact student success. 

• not sure 

• Not sure, We are looking in the possibility of using WorkKeys assessment to help with 
standardization of results. 

• Comparison with state and national scores 

• the more detailed the better 

• My question is do the National Normed assessments really match what our DOE says is 
the competencies to be teaching. 

Teachers 

• I think they are only one element that should be used to measure a student's success. We 
tend to focus "too much" time on standardized tests of any format. 

• I think they should be used as a guide to decision making, not a must. There is too much 
emphasis placed on a standardized score. It should only be used a guide. Schools should 
be given training on how best to incorporate the interpretation of the scores into their 
teaching objectives. 

• This could be an entire training. 

• Some technical assessment data is relevant. However, many times tests and test scores 
are not really good indicators of student learning. NOCTI has a performance component 
to the assessment which is more valid than just taking a multiple choice test. 

• They were hard to understand... 

• I am happy with what I have. 

• Access to the actual test questions or samples, results by question or standard, and an 
easier way to compile results. 
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• Valid content. Questions/answers made by persons that actually work in the trades or 
have a great deal of past(recent) experience in the trade for which questions are being 
written. 

• By keeping them updated and more relevant to industry standards. 

• Expand assessments beyond multiple choice questions 

• The terminology could be directed in a way that is easier to understand. 

• Make them program- specific. 

• Our information is sufficient for decision making. 

• I am unsure at this time 

• I am not sure, it seem to me that students are not interested in working to get better. 



Section III: Professional Development Opportunities 



32. Did your teacher training or administrator college programs include coursework about 
assessment and/or evaluation? 





Administrators 


Teachers 


Yes 


72.9% 


69.2% 


No 


12.5% 


10.3% 


Don't Remember 


14.6% 


15.4% 


N/A — used alternative license to enter teaching with no 
pre-service program 


0.0% 


2.6% 



33. Was it required or optional? 





Administrators 


Teachers 


Required 


64.6% 


69.2% 


Optional 


4.2% 


5.1% 


Don’t remember 


14.6% 


5.1% 



34. What topics were covered in the assessment coursework? (Check all that apply.) 





Administrators 


Teachers 


Don’t remember the topics. 


10.4% 


12.8% 


Different types of assessment methods 


56.3% 


53.8% 


How to decide what should be assessed 


47.9% 


38.5% 


How to write assessment items 


45.8% 


48.7% 


How to interpret assessment data 


64.6% 


35.9% 


How to make instructional adjustments based on assessment 
data 


41.7% 


35.9% 


How to engage others in data analysis and data-based 
decisions 


18.8% 


28.2% 


Other 


4.2% 


2.6% 
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35. If you received professional development in the last five years, what topic(s) did it involve? 
(Check all that apply.) 





Administrators 


Teachers 


Technical training related to CTE area of expertise 


45.8% 


46.2% 


Teaching strategies 


66.7% 


82.1% 


Lesson plan development 


27.1% 


43.6% 


Curriculum development and planning 


64.6% 


64.1% 


Diversity or cultural training 


43.8% 


48.7% 


Classroom management techniques 


54.2% 


56.4% 


Motivating students to learn 


47.9% 


59.0% 


Learning styles 


62.5% 


51.3% 


Legal or ethical issues related to teaching/education 


70.8% 


46.2% 


School or district policies and procedures 


70.8% 


35.9% 


Working with students with special needs 


60.4% 


41.0% 


How to write good tests 


16.7% 


23.1% 


How to use and interpret test data 


43.8% 


23.1% 


Aligning curriculum to national, state, or local 
standards 


56.3% 


64.1% 


Other 


4.2% 


5.1% 



36. If you have received professional development in interpretation and use of technical 
assessment data, which of the following best describes how suitable it was for you and your state 
of readiness? 





Administrators 


Teachers 


I did not have this training 


31.3% 


35.9% 


The training contained the information I needed and was 
presented at an appropriate level. 


39.8% 


38.5% 


The training was too complex and not well explained. 


10.4% 


5.1% 


The training was too low level — I already knew most of what 
was presented. 


8.3% 


0.0% 


The training did not contain enough information about real 
world applications. 


8.3% 


7.7% 


Other 


2.1% 


0.0% 



37. Who provided this professional development in using assessment data? 





Administrators 


Teachers 


A school, district, or state administrator 


14.6% 


15.4% 


A school, district, or state data specialist 


14.6% 


2.6% 


A knowledgeable teacher or peer (inside or outside the 
district) 


2.1% 


15.4% 


A consultant 


12.5% 


5.1% 






















































































An instructor from a local university 


14.6% 


10.3% 


A representative from a professional testing organization 


12.5% 


2.6% 


Other 


6.3% 


7.7% 


Not sure 


8.3% 


17.9% 



38. Do you know how to obtain or access professional development in the use of assessment data 
for data-based decision making? 





Administrators 


Teachers 


Yes 


58.3% 


48.7% 


No 


12.5% 


17.9% 


Not sure 


25.0% 


20.5% 


This would not be my role. 


2.1% 


10.3% 



39. If you were to receive professional development in the use of assessment data for data-based 
decision making, what forms of training would you prefer? (Rank the top three forms with 1 the 
first priority, 2 the second priority, and 3 the third priority.) 





First 


Second 


Third 


Conference presentation(s) 


23.1% 


12.8% 


12.8% 


Web seminars 


7.7% 


7.7% 


7.7% 


Online training modules 


10.3% 


10.3% 


7.7% 


State summer institute workshop 


7.7% 


5.1% 


12.8% 


Train-the-trainer workshop for teacher leaders 


10.3% 


7.7% 


7.7% 


College course 


2.6% 


5.1% 


2.6% 


School-level workshop 


20.5% 


10.3% 


10.3% 


Mentoring or coaching 


7.7% 


0.0% 


2.6% 


Study/support group 


0.0% 


17.9% 


7.7% 


Self-generated (e.g., books, magazine articles on 
topic) 


7.7% 


7.7% 


10.3% 


Other 


0.0% 


0.0% 


0.0% 
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40. Have you participated in inservice workshops on the following topics in the past five years? 



(Check all cells that apply.) 





Administrators 


Teachers 




Yes 


No, but 
they 
were 
available. 


No, they 
were not 
available, 
but I 

wish they 
were. 


Yes 


No, but 
they 
were 
available. 


No, they 
were not 
available, 
but I 

wish they 
were. 


Information on how tests 
are developed and what 
makes a good vs. poor test 


25.0% 


25.0% 


37.5% 


30.8% 


10.3% 


46.2% 


Information on appropriate 
and inappropriate uses of 
tests and test data 


39.6% 


12.5% 


29.2% 


25.6% 


12.8% 


46.2% 


What questions test data 
can and cannot answer 


22.9% 


20.8% 


33.3% 


20.5% 


12.8% 


53.8% 




Yes 


No, but 
they 
were 
available. 


No, they 
were not 
available, 
but I 

wish they 
were. 


Yes 


No, but 
they 
were 
available. 


No, they 
were not 
available, 
but I 

wish they 
were. 


How to interpret student 
level test data (e.g., 
determining student 
strengths and weaknesses, 
determining student 
improvement over time) 


45.8% 


12.5% 


25.0% 


33.3% 


10.3% 


41.0% 


How to interpret group- 
level test data (e.g., shared 
strengths and weaknesses 
within a classroom, 
comparing results to 
classroom practices) 


41.7% 


12.5% 


27.1% 


28.2% 


10.3% 


46.2% 


How to compare classroom 
or individual data to school, 
district, state, or national 
averages 


41.7% 


12.5% 


27.1% 


28.2% 


12.8% 


43.6% 


How to measure student 
and classroom 
improvement over time 


41.7% 


8.3% 


31.3% 


43.6% 


5.1% 


41.0% 


The meaning of technical 
terms used on tests (e.g.. 


35.4% 


14.6% 


29.2% 


38.5% 


2.6% 


46.2% 












































































norms, mean, standard 
deviation, percentage, 
percentile, cut score) 














Information on types of 
tests and test items 
available 


41.7% 


18.8% 


22.9% 


35.9% 


7.7% 


41.0% 


How to select the most 
appropriate measure for the 
curriculum 


27.1% 


14.6% 


35.4% 


25.6% 


12.8% 


46.2% 


Other 


2.1% 


4.2% 


0.0% 


0.0% 


0.0% 


12.8% 



41 If you were to receive professional development in the use of assessment data for data-based 
decision making, who would you most like to deliver the training and why? 





Administrators 


Teachers 


An administrator 


8.3% 


2.6% 


A school, district, or state data specialist 


20.8% 


5.1% 


A knowledgeable teacher or peer 


18.8% 


53.8% 


A consultant 


16.7% 


12.8% 


An instructor from a local university 


0.0% 


2.6% 


A representative from a professional testing organization 


16.7% 


12.8% 


Other 


8.3% 


2.6% 


Not sure 


8.3% 


5.1% 



42. Please explain briefly why you would want that person to deliver the training. 

Administrators 

I would want a knowledgeable administrator to deliver it because 
that would be a peer who has the same perspective I have as a 
fellow administrator. 

I hope they would be able to put it in terms I would understand. An 
unfortunate part of state standardized testing is that the mathematics 
used for federal accountability purposes has changed more than 
An administrator once, and it is somewhat complex. 

Knowledge of recent applications 

State is who we report to — they are best suited to tell us what they 
want. 

They are most likely to stay up to date with what is out there and 
what we need. 

They would have the best overall understanding of the assessments 
that are used in the CareerTech System. 



A school, district, or state data 
specialist 
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Someone local or familiar with our State requirements and needs 
can be interpret the data as we would need it or be able to use for 
program improvement. 

I would feel less intimidated about asking questions. 

I believe that someone with "hands-on" experience in this process 
would know what my concerns would be— and how to relate the 
training in a way that I could benefit. 

What someone with the practical experience 

Person who actually uses and interprets the data 

A knowledgeable teacher or 
peer 

A consultant is usually a specialist in the area that has dealt with 
numerous groups addressing the same issue. The consultant is not 
tied to any specific interest group and is focused on information. 

They are usually skilled in presenting to different groups. 

I think a consultant would be unbiased, as opposed to a rep from a 
testing agency. I feel someone from outside the district can seem 
more credible. 

They would be the expert 

I feel someone from outside the district can seem more credible 
Knowledgable and unbiased 

A consultant 

An instructor from a local 
university 

If we choose a nationally normed test product (commercially based 
test), then that company needs to provide the professional 
development for that product. 

It doesn't matter who as long as they are knowledgeable and can 
present the needed information. 

A representative from a I would want an expert in the field. Someone who works with data 

professional testing on a regular basis. 

organization 

Other: 
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Teacher coaches 


The staff trust them, they are teachers too. 


Someone who can answer 
questions 


Often times presenters used a canned speech and do not get to the 
real need of the audience 


Someone who is actually using 
it and has considerable 
firsthand experience using it. I 
have had it with professional 
presenters who have little 
background 


I am sick to death of the retired super or principal that is now the 
expert on everything and can present on any topic. How stupid does 
the audience need to be to buy this hogwash. 


DESE Program Administration 


He is the one I must report to and ask questions of. 



Teachers 



I am training to be an administrator and a current administrator 
An administrator would best know how to present the data. 

I believe a state data specialist would be better because they know 
the assessments at the state level better. 

A school, district, or state data They become a great source of follow up information 

specialist 

That is a tough call, but I think an experienced teacher with solid 
credentials will gain the respect of the teachers. The experienced 
teacher is in the trenches and practicing what they are 
preaching.... at least I hope! 

They would be able to relate to what we are experiencing in the 
classroom and tailor information for our use. 

A knowledgeable peer has used this data in a manner I would need 
to use it and would be viewing interpretation in much the same 
manner as I would. 

Teachers are the people working with the students. Too often 
administrators forget the daily life as a teacher and all that it entails 
and consultants don't have a clue sometimes. They "pretend" to be 
in your shoes to help with a "quick fix". Not all consultants are bad. 
Many do a great job. But when it comes to teachers having to 
implement new teaching strategies, it is much better to have it 
A knowledgeable teacher or delivered by other teachers who are in the classroom doing it, rather 
peer than administrators who see it as the next "big thing" to all conform 
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too to try to increase student achievement.; 



A consultant 

An instructor from a local 
university 



A representative from a 
professional testing 
organization 



Familiar with peer needs 

They know better ways to do what I do. They have all the same 
challenges as I do; They have a more applied knowledge. 

Would have the same point of reference 

They know what teacher's need. They tend not to be too technical 
like some administrators or other experts can be. I think someone 
who uses the data themselves and can relate and apply this 
information in a practical rather than in theoretical manner 

More relaxed with a fellow co-worker. 

Training should be based on subject area. 

A current expert would be best because others are too far removed 
from training. 

They have a more applied knowledge. 

I think someone who uses the data themselves and can relate and 
apply this information in a practical rather than in theoretical 
manner. 

I would want the consultant to possibly be the instructor I had for 
my graduate course. .And he is also a consultant with PATTAN. 



Someone outside the school would provide a different prospective. 




Seems like it makes the best sense. 



They should know there materials and why that is what is assessed. 

Want someone that deals with that subject at a professional level 
and knows how to convey that knowledge to others.; 

Want someone that deals with that subject at a professional level 
and knows how to convey that knowledge to others. 

For better understanding, I prefer the delivery of the training to be 
conducted by the group that developed the assessment data 



189 













Hopefully they would have a good understanding of the subject! 


Other: 




Someone who has documented 
success in this area. This 
person would be able to 
communicate theoretically and 
concretely methods that would 
lead to success and 
improvement. 


We have plenty of people who tell us what can be done for a lesson 
or a day but none who can demonstrate to us the keys to unlocking 
sustained progress. This person may be impossible to find. 



43. Have you assessed the professional development needs of the staff with regard to assessment 
and data-based decision making? 





Administrators 


Yes 


39.6% 


No 


58.3% 



44. For the professional development your teachers received on any topic in the past five years, 



what form did that professional development take? 



Teachers have not participated in professional 
development. 


4.2% 








1 (less 
effective) 


2 


3 (more 
effective) 










Conference presentation(s) 


10.4% 


6.3% 


10.4% 


Web Seminars 


2.1% 


18.3% 


2.1% 


Online training modules 


8.3% 


0.0% 


8.3% 


State summer institute workshop 


6.3% 


6.3% 


12.5% 


Train-the-trainer workshop for teacher leaders 


0.0% 


6.3% 


2.1% 


College course 


2.1% 


4.2% 


4.2% 


School-level workshop 


4.2% 


12.5% 


4.2% 


Mentoring or coaching 


0.0% 


2.1% 


12.5% 


Study group/collaborative team 


10.4% 


4.2% 


12.5% 


Self-generated (e.g., books, magazine articles on topic) 


12.5% 


6.3% 


2.1% 



45. Have your teachers had any professional development in the past five years specifically on 
the topic of interpretation and use of assessment data? 





Administrators 


No 


45.8% 


Yes, most or all of them 


27.1% 
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Yes, some of them 


22.9% 


Yes, but has been more than 5 years 
ago 


2.1% 



46. Do you see a need for your teachers to have training (or additional training) in the use of 
assessment data for data-based decision making? 





Administrators 


Yes 


77.1% 


No 


4.2% 


Not sure 


18.8% 



47. If you wanted to provide your teachers with professional development in the use of 
assessment data for data-based decision making, would you know where or how to obtain or 
access it? 





Administrators 


Yes 


58.3% 


No 


41.7% 



48. If your teachers were to receive professional development in the use of assessment data for 
data-based decision making, what forms of training would you prefer? (Rank the top three forms 
with 1 the first priority, 2 the second priority, and 3 the third priority.) (Note: survey program 
malfunction — only 2 ratings.) 





Administrators 




First 


Second 


Conference presentation(s) 


10.4% 


16.7% 


Web seminars 


4.2% 


6.3% 


Online training modules 


4.2% 


10.4% 


State summer institute workshop 


10.4% 


8.3% 


Train-the-trainer workshop for teacher leaders 


10.4% 


8.3% 


College course 


0.0% 


2.1% 


School-level workshop 


22.9% 


12.5% 


Mentoring or coaching 


10.4% 


12.5% 


Study group/collaborative team 


14.6% 


14.6% 


Self-generated (e.g., books, magazine articles on 
topic) 


4.2% 


0.0% 


Other 


0.0% 


0.0% 



49. What topics would you like the training to contain? (Check all that apply. Also rank the top 
three uses with 1 the first priority, 2 the second priority, and 3 the third priority.) (Note: survey 
program malfunction — no prioritization provided.) 



Administrators 
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Information on how tests are developed and what makes a 
good vs. poor test 


56.3% 


Information on appropriate and inappropriate uses of tests 
and test data 


60.4% 


What questions test data can and cannot answer 


43.8% 


How to interpret student-level test data (e.g., determining 
student strengths and weaknesses, determining student 
improvement over time) 


81.3% 


How to interpret group-level test data (e.g., shared 
strengths and weaknesses within a classroom, comparing 
results to classroom practices) 


58.3% 


How to compare classroom or individual data to school, 
district, state, or national averages 


52.1% 


How to measure student and classroom improvement over 
time 


70.8% 


The meaning of technical terms used on tests (e.g., norms, 
mean, standard deviation, percentage, percentile, cut score) 


39.6% 


Information on types of tests and test items available 


37.5% 


How to select the most appropriate measure for the 
curriculum 


58.3% 


Other 


0.0% 



Section IV: General Assessment Perception 

50. Overall, what is your perception of the value of technical skill assessment? 





Administrators 


Teachers 


Very positive 


37.5% 


25.6% 


Somewhat positive 


45.8% 


46.2% 


Neutral 


12.5% 


17.9% 


Somewhat negative 


4.2% 


2.6% 


Very negative 


0.0% 


2.6% 



51. Has your opinion on the value of standardized technical skill assessment changed over the 
past five years? 





Administrators 


Teachers 


Yes — it has become more positive. 


37.5% 


25.6% 


Yes— it has become more negative. 


4.2% 


7.7% 


No— it is about the same. 


50.0% 


53.8% 


Not sure 


2.1% 


5.1% 


N/A — less than 5 years experience 


6.3% 


2.6% 



52. If your opinion has changed, please briefly describe why. 
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Yes — it has become more positive. 

Administrators 

• In 2000, 1 mandated that NOCTI or a skill certification exam must be taken as an element 
of the Graduation Project for Seniors. Our Board of Directors supported my position that 
if a student did not take it they would not walk for graduation. This survey confirms my 
position.; 

• I have become more confident in the usefulness for continuous improvement. 

• 'I worked with testing data extensively in the past and see how it is extremely beneficial 

• As society and job requirements change, so to does the need to rightly use assessment 
data 

• I believe that tests have improved. If the tests are better and the data is more accurate it is 
worth study. 

• The use of technical skill assessment seems to add accountability to career education for 
the teachers, student, and career center. 

• Opinion has changed because of increased pressures to be more accountable. 

Teachers 

• I have been given a greater understanding through the trainings of the need of assessment 
data and its application to career and tech ed. 

• Recognized trade test should be used ex AWS State boards CAN 

• Identifies areas for improvement in student education; 'The implementation of state 
exams and requiring students to pass for graduation requires a solid teach to the test 
concept. 

• It has become more positive now that I understand it and can use it more. 

• Identifies areas for improvement in student education 

• The implementation of state exams and requiring students to pass for graduation requires 
a solid teach to the test concept. 

Yes — it has become more negative. 

Administrators 

• Because it is difficult to find a national recognized test that is appropriate for every career 
path. 

Teachers 

• Student learning is being compromised to meet AYP and increase student test scores on 
high stakes tests. Students are now required to take remedial classes who have scored 
"low" on state mandated tests and loose out on other classes or electives. 

• Teachers are now forced to teach to the test so that the students pass it, and other 
"excellent" curriculum that is not tested like consumer math, art, music, 
entrepreneurship/business, health/phys-ed are no longer looked upon as "relevant" 
because they are not on the test. However, to become a successful, productive citizen, 
you need more that algebra and reading. Testing with both teachers and students has 
dramatically changed the educational system, not as a way to help students increase 
learning, but a means to determine what schools are doing wrong with students whose 
strength is not testing. 
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• In the future the assessment data results should used as a means of finding ways of 
improving student achievement. 

Section V : Educational Situation and Demographics 

53. (school and district/region name omitted from this summary) 

54. In what state is your school? 





Administrators 


Teachers 


Illinois 


4 


0 


Missouri 


15 


9 


Oklahoma 


9 


11 


Pennsylvania 


10 


17 


Virginia 


10 


1 



55. What CTE program(s) do you teach/administer? (Check one or more.) 





Administrators 


Teachers 


A business cluster program 


0.0% 


0.0% 


A construction cluster program 


87.5% 


28.2% 


A health cluster program 


75.0% 


30.8% 


A manufacturing cluster 
program 


70.8% 


15.4% 



56. Which of the following best describes your highest level of education? 





Administrators 


Teachers 


High school diploma 


0.0% 


0.0% 


High school diploma plus some college courses/credits 


0.0% 


7.7% 


High school diploma plus some specialized training (e.g., 
Military Occupational Specialty, apprenticeship) 


2.1% 


2.6% 


Associate’s degree 


0.0% 


7.7% 


Bachelor’s degree 


0.0% 


30.8% 


Master’s degree 


60.4% 


48.7% 


Specialist’s degree 


20.8% 


0.0% 


Doctorate 


16.7% 


0.0% 



57. How many years have you been in this position in your current school? 





Administrators 


Teachers 


Less than 1 


2.1% 


5.1% 


1 to 5 


52.1% 


28.2% 


6 to 10 


22.9% 


33.3% 


11 to 15 


4.2% 


5.1% 


16 to 20 


6.3% 


7.7% 
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58. What is your gender? 





Administrators 


Teachers 


Female 


31.3% 


51.3% 


Male 


66.7% 


46.2% 



59. What is your age? 





Administrators 


Teachers 


Under 21 


0.0% 


0.0% 


21 to 30 


0.0% 


5.1% 


31 to 40 


8.3% 


28.2% 


41 to 50 


31.3% 


33.3% 


5 1 to 60 


47.9% 


28.2% 


Over 60 


10.4% 


2.6% 



60. How many years have you been teaching? 





Teachers 


Less than 1 


0.0% 


1 to 2 


2.6% 


3 to 5 


15.4% 


6 to 10 


33.3% 


11 to 15 


10.3% 


16 to 20 


5.1% 


More than 20 


30.8% 



61. How did you become licensed or certified to teach in CTE? 





Teachers 


Traditional college/university pre- service 
program 


69.2% 


Alternative licensure route 


20.5% 


Other (please describe) 


7.7% 



62. How many years of experience in the occupation/trade did you have before being licensed to 
teach the CTE program in your state? 

Teacher average: 10.55. Range: 0-24 years 

63. How many years of teaching experience did you have prior to becoming an administrator? 





Administrators 


None 


0.0% 


Less than one 


0.0% 
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1 to 5 


22.9% 


6 to 10 


25.0% 


11 to 15 


27.1% 


More than 15 


25.0% 



64. How many years have you been an administrator? 





Administrators 


Less than 1 


0.0% 


1 to 2 


12.5% 


3 to 5 


16.7% 


6 to 10 


29.2% 


11 to 15 


16.7% 


16 to 20 


10.4% 


More than 20 


14.6% 



65. How many years of teaching experience did you have specifically in CTE prior to becoming 
an administrator? 





Administrators 


None 


29.2% 


Less than one 


2.1% 


1 to 5 


20.8% 


6 to 10 


16.7% 


11 to 15 


16.7% 


More than 15 


14.6% 
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Appendix 3B 



Material Review Protocol 

Professional Development for Teachers and Administrators 
on the Use of Assessment Data 
Year III NOCTI-NRCCTE 
Review Form for Workshop Materials 

We appreciate your willingness to help us refine the workshop materials, which consist of a 
Powerpoint presentation and a Facilitator’s Guide. Note that the materials for the participants 
will be like the Facilitator’s Guide, except that pages 5-10 will be deleted as will the blue 
suggested answers. Also, graphics and some informal but professional-level humor will be added 
later on. 

Please use the following form to enter your comments, and feel free to expand the form where 
needed. After you are finished, please email the form to sandvpritz@nocti.org . We hope to 
receive your input by October 26. 



Section and subsection 


Fine 
as is 


Not 

needed 


Meaning unclear 
or Not phrased 
appropriately 


Suggestions for 
change/ rationale 


PowerPoint Presentation 

(slide # as subsection for 
comment) 


























































































Facilitator Guide 










Summary of the NOCTI- 
NRCCTE Project 


















































The Workshop 
Presentation 










• Step 1 










• Step 2 










• Step 3 
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• Step 4 










• Step 5 










Social Networking 










Worksheets/content Stepl 










Worksheets/content Step 2 










Worksheets/content Step 3 










Worksheets/content Step 4 










Glossary 











Opinion and comments on the overall package: 
• Suitable for a one-day workshop? 



• Targeted at the right level for secondary CTE administrators and teachers? 

• Appropriate tone? 

• Anything else? 

We are very grateful for your input! 
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